2024-04-26T04:17:49.433 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T04:17:49.434 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --name yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537 --description rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} --verbose -- /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/orig.config.yaml 2024-04-26T04:17:49.473 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T04:17:49.551 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537 branch: wip-yuri2-testing-2024-04-24-0914-squid description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} email: yweinste@redhat.com first_in_suite: false job_id: '7674537' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-yuri2-testing-2024-04-24-0914-squid ceph: conf: global: mon election default strategy: 3 osd debug inject dispatch delay duration: 0.1 osd debug inject dispatch delay probability: 0.1 osd_async_recovery_min_cost: 1 osd_object_clean_region_max_num_intervals: 1000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.1 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - slow request - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED sha1: ddeaa91797339a19172c3036ff48cca58c12f448 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: ddeaa91797339a19172c3036ff48cca58c12f448 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri2-testing-2024-04-24-0914-squid sha1: ddeaa91797339a19172c3036ff48cca58c12f448 owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 1848 sha1: ddeaa91797339a19172c3036ff48cca58c12f448 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri2-testing-2024-04-24-0914-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_ddeaa91797339a19172c3036ff48cca58c12f448/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: ddeaa91797339a19172c3036ff48cca58c12f448 targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOpPyOBmkp02ua0j8uAGE6mVom8u8MxXYf8hD9EdS8KvAiHHjzMROwJtse/ELhFk77iWt7NO4IUcqGpl0EyUBQ0= smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDtaKlZyDmKaDMPdgJlVIfpH2fxSXBxeeltzobA7Qh+t5mQBLUto9ttQ8myJEXBOqOZN/3rCBHUaAv3Gjf5n7qA= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNY+ds4buWbZuY9lv3VjkNF/ay4yilCJKIAfwijdXptA/UX6nyKxz05LNhuN11GOW2ZzgYckEYKvXqECqcH23Vs= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLwLyDbrNL+/xuKJZRbJCPASorcyTzEoDK8CiexqrA/fptcNO/IYz2WZAnTIz2mNwNMJhTvehJaC2sJpcNJJSgw= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: bluestore_new_sharding: random chance_bluestore_reshard: 1 chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 min_in: 8 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: c: 2 crush-failure-domain: osd k: 4 m: 3 name: shecprofile plugin: shec objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 400 write_append_excl: false teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_01:23:56 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T04:17:49.551 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_ddeaa91797339a19172c3036ff48cca58c12f448/qa; will attempt to use it 2024-04-26T04:17:49.552 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_ddeaa91797339a19172c3036ff48cca58c12f448/qa/tasks 2024-04-26T04:17:49.552 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T04:17:49.553 INFO:teuthology.task.internal:Checking packages... 2024-04-26T04:17:49.578 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'ddeaa91797339a19172c3036ff48cca58c12f448' 2024-04-26T04:17:49.578 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T04:17:49.578 INFO:teuthology.packaging:ref: None 2024-04-26T04:17:49.578 INFO:teuthology.packaging:tag: None 2024-04-26T04:17:49.578 INFO:teuthology.packaging:branch: wip-yuri2-testing-2024-04-24-0914-squid 2024-04-26T04:17:49.578 INFO:teuthology.packaging:sha1: ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T04:17:49.579 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-yuri2-testing-2024-04-24-0914-squid 2024-04-26T04:17:49.809 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:17:49.810 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T04:17:49.817 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T04:17:49.817 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T04:17:49.837 INFO:teuthology.task.internal:Saving configuration 2024-04-26T04:17:49.853 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T04:17:49.869 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T04:17:49.894 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 04:11:40.071010', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEMBpOJnofZd/5QLcZhgpjZ6dP2FwifSyXSw9baKWLYi'} 2024-04-26T04:17:49.918 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 04:11:40.065631', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKQkHpaCG+5XiFseRn/M3SssMz2WTghNjleA3qhjygB4'} 2024-04-26T04:17:49.941 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 04:11:40.069328', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIP5vGep+oi2jzk9O7VxXajOc+0fGYojsUTofCz4B4nY6'} 2024-04-26T04:17:49.963 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 04:11:40.067597', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGCp8b1CDPtRHf1lrwT0FZnujKLUOizGBZYzyHaNXs9x'} 2024-04-26T04:17:49.964 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T04:17:50.230 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2024-04-26T04:17:50.230 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2024-04-26T04:17:50.230 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2024-04-26T04:17:50.230 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2024-04-26T04:17:50.230 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T04:17:50.376 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3eaa22a3a0>, signals=[15]) 2024-04-26T04:17:50.376 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T04:17:50.382 INFO:teuthology.task.internal:Opening connections... 2024-04-26T04:17:50.383 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T04:17:50.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:17:50.466 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2024-04-26T04:17:50.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:17:50.545 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-04-26T04:17:50.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:17:50.630 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-04-26T04:17:50.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:17:50.707 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T04:17:50.715 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-04-26T04:17:50.720 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-04-26T04:17:50.721 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-04-26T04:17:50.766 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-26T04:17:50.767 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Ubuntu" 2024-04-26T04:17:50.767 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="22.04" 2024-04-26T04:17:50.767 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-26T04:17:50.768 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_CODENAME=jammy 2024-04-26T04:17:50.768 INFO:teuthology.orchestra.run.smithi012.stdout:ID=ubuntu 2024-04-26T04:17:50.768 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE=debian 2024-04-26T04:17:50.768 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-26T04:17:50.768 INFO:teuthology.orchestra.run.smithi012.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-26T04:17:50.768 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-26T04:17:50.768 INFO:teuthology.orchestra.run.smithi012.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-26T04:17:50.768 INFO:teuthology.orchestra.run.smithi012.stdout:UBUNTU_CODENAME=jammy 2024-04-26T04:17:50.769 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-04-26T04:17:50.793 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2024-04-26T04:17:50.799 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2024-04-26T04:17:50.799 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2024-04-26T04:17:50.846 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-26T04:17:50.846 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="Ubuntu" 2024-04-26T04:17:50.846 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="22.04" 2024-04-26T04:17:50.846 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-26T04:17:50.846 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_CODENAME=jammy 2024-04-26T04:17:50.846 INFO:teuthology.orchestra.run.smithi113.stdout:ID=ubuntu 2024-04-26T04:17:50.846 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE=debian 2024-04-26T04:17:50.846 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-26T04:17:50.846 INFO:teuthology.orchestra.run.smithi113.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-26T04:17:50.847 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-26T04:17:50.847 INFO:teuthology.orchestra.run.smithi113.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-26T04:17:50.847 INFO:teuthology.orchestra.run.smithi113.stdout:UBUNTU_CODENAME=jammy 2024-04-26T04:17:50.847 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2024-04-26T04:17:50.868 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-04-26T04:17:50.874 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-04-26T04:17:50.874 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-04-26T04:17:50.923 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-26T04:17:50.923 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="Ubuntu" 2024-04-26T04:17:50.923 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="22.04" 2024-04-26T04:17:50.923 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-26T04:17:50.923 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_CODENAME=jammy 2024-04-26T04:17:50.923 INFO:teuthology.orchestra.run.smithi137.stdout:ID=ubuntu 2024-04-26T04:17:50.923 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE=debian 2024-04-26T04:17:50.923 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-26T04:17:50.923 INFO:teuthology.orchestra.run.smithi137.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-26T04:17:50.923 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-26T04:17:50.923 INFO:teuthology.orchestra.run.smithi137.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-26T04:17:50.923 INFO:teuthology.orchestra.run.smithi137.stdout:UBUNTU_CODENAME=jammy 2024-04-26T04:17:50.924 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-04-26T04:17:50.942 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-04-26T04:17:50.948 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-04-26T04:17:50.948 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-04-26T04:17:51.000 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-26T04:17:51.000 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Ubuntu" 2024-04-26T04:17:51.000 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="22.04" 2024-04-26T04:17:51.000 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-26T04:17:51.000 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_CODENAME=jammy 2024-04-26T04:17:51.000 INFO:teuthology.orchestra.run.smithi178.stdout:ID=ubuntu 2024-04-26T04:17:51.000 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE=debian 2024-04-26T04:17:51.000 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-26T04:17:51.000 INFO:teuthology.orchestra.run.smithi178.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-26T04:17:51.000 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-26T04:17:51.001 INFO:teuthology.orchestra.run.smithi178.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-26T04:17:51.001 INFO:teuthology.orchestra.run.smithi178.stdout:UBUNTU_CODENAME=jammy 2024-04-26T04:17:51.001 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-04-26T04:17:51.018 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T04:17:51.028 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T04:17:51.035 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T04:17:51.036 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-04-26T04:17:51.038 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2024-04-26T04:17:51.041 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-04-26T04:17:51.043 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-04-26T04:17:51.048 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T04:17:51.054 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T04:17:51.054 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-04-26T04:17:51.087 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2024-04-26T04:17:51.090 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-04-26T04:17:51.092 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-04-26T04:17:51.109 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T04:17:51.204 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T04:17:51.218 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-26T04:17:51.218 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-26T04:17:51.219 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T04:17:51.219 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T04:17:51.220 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T04:17:51.220 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T04:17:51.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:17:51.225 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2024-04-26T04:17:51.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:17:51.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:17:51.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:17:51.227 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-04-26T04:17:51.227 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-04-26T04:17:51.228 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-04-26T04:17:51.270 INFO:teuthology.orchestra.run.smithi113.stdout:5.15.0-56-generic 2024-04-26T04:17:51.270 INFO:teuthology.task.kernel:Running kernel on smithi113: 5.15.0-56-generic 2024-04-26T04:17:51.270 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-get clean 2024-04-26T04:17:51.274 INFO:teuthology.orchestra.run.smithi012.stdout:5.15.0-56-generic 2024-04-26T04:17:51.274 INFO:teuthology.task.kernel:Running kernel on smithi012: 5.15.0-56-generic 2024-04-26T04:17:51.274 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2024-04-26T04:17:51.276 INFO:teuthology.orchestra.run.smithi137.stdout:5.15.0-56-generic 2024-04-26T04:17:51.276 INFO:teuthology.task.kernel:Running kernel on smithi137: 5.15.0-56-generic 2024-04-26T04:17:51.276 DEBUG:teuthology.orchestra.run.smithi137:> sudo apt-get clean 2024-04-26T04:17:51.277 INFO:teuthology.orchestra.run.smithi178.stdout:5.15.0-56-generic 2024-04-26T04:17:51.277 INFO:teuthology.task.kernel:Running kernel on smithi178: 5.15.0-56-generic 2024-04-26T04:17:51.277 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get clean 2024-04-26T04:17:51.400 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-04-26T04:17:51.407 DEBUG:teuthology.orchestra.run.smithi137:> sudo apt-get update 2024-04-26T04:17:51.429 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-get update 2024-04-26T04:17:51.433 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get update 2024-04-26T04:17:51.592 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-26T04:17:51.603 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-26T04:17:51.640 INFO:teuthology.orchestra.run.smithi113.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-26T04:17:51.643 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-26T04:17:51.643 INFO:teuthology.orchestra.run.smithi178.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-26T04:17:51.649 INFO:teuthology.orchestra.run.smithi137.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-26T04:17:51.652 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-26T04:17:51.656 INFO:teuthology.orchestra.run.smithi137.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-26T04:17:51.660 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-26T04:17:51.662 INFO:teuthology.orchestra.run.smithi178.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-26T04:17:51.712 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-26T04:17:51.783 INFO:teuthology.orchestra.run.smithi113.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-26T04:17:51.826 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-26T04:17:51.830 INFO:teuthology.orchestra.run.smithi137.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-26T04:17:51.860 INFO:teuthology.orchestra.run.smithi178.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-26T04:17:51.896 INFO:teuthology.orchestra.run.smithi178.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-26T04:17:51.904 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-26T04:17:51.907 INFO:teuthology.orchestra.run.smithi137.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-26T04:17:51.976 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-26T04:17:51.989 INFO:teuthology.orchestra.run.smithi113.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-26T04:17:52.055 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-26T04:17:52.077 INFO:teuthology.orchestra.run.smithi137.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-26T04:17:52.098 INFO:teuthology.orchestra.run.smithi137.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-26T04:17:52.114 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-26T04:17:52.163 INFO:teuthology.orchestra.run.smithi113.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-26T04:17:52.166 INFO:teuthology.orchestra.run.smithi178.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-26T04:17:52.181 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-26T04:17:52.227 INFO:teuthology.orchestra.run.smithi113.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-26T04:17:52.261 INFO:teuthology.orchestra.run.smithi137.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-26T04:17:52.262 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-26T04:17:52.262 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-26T04:17:52.268 INFO:teuthology.orchestra.run.smithi137.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-26T04:17:52.268 INFO:teuthology.orchestra.run.smithi137.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-26T04:17:52.296 INFO:teuthology.orchestra.run.smithi113.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-26T04:17:52.299 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-26T04:17:52.299 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-26T04:17:52.306 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-26T04:17:52.306 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-26T04:17:52.320 INFO:teuthology.orchestra.run.smithi137.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-26T04:17:52.321 INFO:teuthology.orchestra.run.smithi137.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-26T04:17:52.323 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-26T04:17:52.326 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,071 kB] 2024-04-26T04:17:52.328 INFO:teuthology.orchestra.run.smithi137.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-26T04:17:52.329 INFO:teuthology.orchestra.run.smithi137.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-26T04:17:52.382 INFO:teuthology.orchestra.run.smithi137.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-26T04:17:52.412 INFO:teuthology.orchestra.run.smithi137.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,071 kB] 2024-04-26T04:17:52.418 INFO:teuthology.orchestra.run.smithi178.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-26T04:17:52.424 INFO:teuthology.orchestra.run.smithi113.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-26T04:17:52.435 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-26T04:17:52.437 INFO:teuthology.orchestra.run.smithi137.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-26T04:17:52.441 INFO:teuthology.orchestra.run.smithi178.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-26T04:17:52.442 INFO:teuthology.orchestra.run.smithi178.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-26T04:17:52.454 INFO:teuthology.orchestra.run.smithi113.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-26T04:17:52.454 INFO:teuthology.orchestra.run.smithi113.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-26T04:17:52.454 INFO:teuthology.orchestra.run.smithi113.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-26T04:17:52.458 INFO:teuthology.orchestra.run.smithi137.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-26T04:17:52.459 INFO:teuthology.orchestra.run.smithi137.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-26T04:17:52.461 INFO:teuthology.orchestra.run.smithi137.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-26T04:17:52.461 INFO:teuthology.orchestra.run.smithi137.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-26T04:17:52.462 INFO:teuthology.orchestra.run.smithi137.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-26T04:17:52.463 INFO:teuthology.orchestra.run.smithi137.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-26T04:17:52.464 INFO:teuthology.orchestra.run.smithi137.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-26T04:17:52.477 INFO:teuthology.orchestra.run.smithi178.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-26T04:17:52.478 INFO:teuthology.orchestra.run.smithi178.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-26T04:17:52.479 INFO:teuthology.orchestra.run.smithi137.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-26T04:17:52.484 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-26T04:17:52.485 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-26T04:17:52.487 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-26T04:17:52.487 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-26T04:17:52.488 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-26T04:17:52.489 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-26T04:17:52.490 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-26T04:17:52.491 INFO:teuthology.orchestra.run.smithi178.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-26T04:17:52.492 INFO:teuthology.orchestra.run.smithi178.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-26T04:17:52.492 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-26T04:17:52.492 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-26T04:17:52.504 INFO:teuthology.orchestra.run.smithi178.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,071 kB] 2024-04-26T04:17:52.508 INFO:teuthology.orchestra.run.smithi113.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-26T04:17:52.519 INFO:teuthology.orchestra.run.smithi113.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-26T04:17:52.519 INFO:teuthology.orchestra.run.smithi113.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-26T04:17:52.530 INFO:teuthology.orchestra.run.smithi178.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-26T04:17:52.530 INFO:teuthology.orchestra.run.smithi137.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-26T04:17:52.530 INFO:teuthology.orchestra.run.smithi137.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-26T04:17:52.531 INFO:teuthology.orchestra.run.smithi137.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-26T04:17:52.532 INFO:teuthology.orchestra.run.smithi137.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-26T04:17:52.532 INFO:teuthology.orchestra.run.smithi137.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-26T04:17:52.538 INFO:teuthology.orchestra.run.smithi178.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-26T04:17:52.538 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-26T04:17:52.538 INFO:teuthology.orchestra.run.smithi178.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-26T04:17:52.539 INFO:teuthology.orchestra.run.smithi178.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-26T04:17:52.542 INFO:teuthology.orchestra.run.smithi113.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-26T04:17:52.549 INFO:teuthology.orchestra.run.smithi137.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-26T04:17:52.551 INFO:teuthology.orchestra.run.smithi113.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-26T04:17:52.553 INFO:teuthology.orchestra.run.smithi113.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-26T04:17:52.561 INFO:teuthology.orchestra.run.smithi178.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-26T04:17:52.562 INFO:teuthology.orchestra.run.smithi137.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-26T04:17:52.562 INFO:teuthology.orchestra.run.smithi178.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-26T04:17:52.562 INFO:teuthology.orchestra.run.smithi137.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-26T04:17:52.563 INFO:teuthology.orchestra.run.smithi178.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-26T04:17:52.564 INFO:teuthology.orchestra.run.smithi178.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-26T04:17:52.564 INFO:teuthology.orchestra.run.smithi137.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-26T04:17:52.565 INFO:teuthology.orchestra.run.smithi178.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-26T04:17:52.568 INFO:teuthology.orchestra.run.smithi113.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-26T04:17:52.568 INFO:teuthology.orchestra.run.smithi113.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-26T04:17:52.569 INFO:teuthology.orchestra.run.smithi113.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-26T04:17:52.570 INFO:teuthology.orchestra.run.smithi113.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-26T04:17:52.570 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-26T04:17:52.570 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-26T04:17:52.570 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-26T04:17:52.571 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-26T04:17:52.585 INFO:teuthology.orchestra.run.smithi178.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-26T04:17:52.586 INFO:teuthology.orchestra.run.smithi178.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-26T04:17:52.587 INFO:teuthology.orchestra.run.smithi178.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-26T04:17:52.587 INFO:teuthology.orchestra.run.smithi178.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-26T04:17:52.587 INFO:teuthology.orchestra.run.smithi178.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-26T04:17:52.612 INFO:teuthology.orchestra.run.smithi113.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-26T04:17:52.618 INFO:teuthology.orchestra.run.smithi178.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-26T04:17:52.646 INFO:teuthology.orchestra.run.smithi137.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-26T04:17:52.647 INFO:teuthology.orchestra.run.smithi178.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-26T04:17:52.654 INFO:teuthology.orchestra.run.smithi137.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-26T04:17:52.654 INFO:teuthology.orchestra.run.smithi137.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-26T04:17:52.655 INFO:teuthology.orchestra.run.smithi113.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-26T04:17:52.667 INFO:teuthology.orchestra.run.smithi178.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-26T04:17:52.669 INFO:teuthology.orchestra.run.smithi137.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-26T04:17:52.671 INFO:teuthology.orchestra.run.smithi113.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-26T04:17:52.671 INFO:teuthology.orchestra.run.smithi113.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-26T04:17:52.689 INFO:teuthology.orchestra.run.smithi178.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-26T04:17:52.739 INFO:teuthology.orchestra.run.smithi113.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-26T04:17:52.741 INFO:teuthology.orchestra.run.smithi113.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-26T04:17:52.750 INFO:teuthology.orchestra.run.smithi113.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-26T04:17:52.762 INFO:teuthology.orchestra.run.smithi178.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-26T04:17:52.788 INFO:teuthology.orchestra.run.smithi178.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-26T04:17:52.809 INFO:teuthology.orchestra.run.smithi178.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-26T04:17:52.839 INFO:teuthology.orchestra.run.smithi178.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-26T04:17:52.852 INFO:teuthology.orchestra.run.smithi137.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-26T04:17:52.873 INFO:teuthology.orchestra.run.smithi178.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-26T04:17:52.897 INFO:teuthology.orchestra.run.smithi178.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-26T04:17:52.915 INFO:teuthology.orchestra.run.smithi113.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-26T04:17:52.917 INFO:teuthology.orchestra.run.smithi178.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-26T04:17:52.939 INFO:teuthology.orchestra.run.smithi178.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-26T04:17:52.966 INFO:teuthology.orchestra.run.smithi178.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-26T04:17:52.986 INFO:teuthology.orchestra.run.smithi178.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-26T04:17:53.112 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-26T04:17:53.150 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-26T04:17:53.156 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-26T04:17:53.262 INFO:teuthology.orchestra.run.smithi137.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-26T04:17:53.268 INFO:teuthology.orchestra.run.smithi137.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-26T04:17:53.282 INFO:teuthology.orchestra.run.smithi137.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-26T04:17:53.283 INFO:teuthology.orchestra.run.smithi137.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-26T04:17:53.285 INFO:teuthology.orchestra.run.smithi137.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-26T04:17:53.328 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-26T04:17:53.329 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-26T04:17:53.346 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-26T04:17:53.347 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-26T04:17:53.388 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-26T04:17:53.504 INFO:teuthology.orchestra.run.smithi113.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,071 kB] 2024-04-26T04:17:53.615 INFO:teuthology.orchestra.run.smithi113.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-26T04:17:53.637 INFO:teuthology.orchestra.run.smithi113.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-26T04:17:53.637 INFO:teuthology.orchestra.run.smithi113.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-26T04:17:53.641 INFO:teuthology.orchestra.run.smithi113.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-26T04:17:53.641 INFO:teuthology.orchestra.run.smithi113.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-26T04:17:53.642 INFO:teuthology.orchestra.run.smithi113.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-26T04:17:53.642 INFO:teuthology.orchestra.run.smithi113.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-26T04:17:53.647 INFO:teuthology.orchestra.run.smithi113.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-26T04:17:53.651 INFO:teuthology.orchestra.run.smithi113.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-26T04:17:54.428 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-26T04:17:54.434 INFO:teuthology.orchestra.run.smithi113.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-26T04:17:54.517 INFO:teuthology.orchestra.run.smithi113.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-26T04:17:54.600 INFO:teuthology.orchestra.run.smithi113.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-26T04:17:54.685 INFO:teuthology.orchestra.run.smithi113.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-26T04:17:54.718 INFO:teuthology.orchestra.run.smithi113.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-26T04:17:54.857 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-26T04:17:54.864 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-26T04:17:54.910 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-26T04:17:54.911 INFO:teuthology.orchestra.run.smithi012.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-26T04:17:54.920 INFO:teuthology.orchestra.run.smithi012.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-26T04:17:58.027 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 13.2 MB in 2s (6,040 kB/s) 2024-04-26T04:17:58.049 INFO:teuthology.orchestra.run.smithi137.stdout:Fetched 13.2 MB in 2s (6,027 kB/s) 2024-04-26T04:17:59.063 INFO:teuthology.orchestra.run.smithi113.stdout:Fetched 13.2 MB in 3s (4,118 kB/s) 2024-04-26T04:17:59.332 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-04-26T04:17:59.351 DEBUG:teuthology.orchestra.run.smithi178:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-26T04:17:59.357 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-04-26T04:17:59.373 DEBUG:teuthology.orchestra.run.smithi137:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-26T04:17:59.411 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-04-26T04:17:59.432 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-04-26T04:17:59.462 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 13.2 MB in 3s (3,836 kB/s) 2024-04-26T04:17:59.603 INFO:teuthology.orchestra.run.smithi178.stdout:Building dependency tree... 2024-04-26T04:17:59.604 INFO:teuthology.orchestra.run.smithi178.stdout:Reading state information... 2024-04-26T04:17:59.632 INFO:teuthology.orchestra.run.smithi137.stdout:Building dependency tree... 2024-04-26T04:17:59.632 INFO:teuthology.orchestra.run.smithi137.stdout:Reading state information... 2024-04-26T04:17:59.776 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages were automatically installed and are no longer required: 2024-04-26T04:17:59.777 INFO:teuthology.orchestra.run.smithi178.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-26T04:17:59.777 INFO:teuthology.orchestra.run.smithi178.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-26T04:17:59.777 INFO:teuthology.orchestra.run.smithi178.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-26T04:17:59.777 INFO:teuthology.orchestra.run.smithi178.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-26T04:17:59.777 INFO:teuthology.orchestra.run.smithi178.stdout: python2.7-minimal 2024-04-26T04:17:59.778 INFO:teuthology.orchestra.run.smithi178.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-26T04:17:59.779 INFO:teuthology.orchestra.run.smithi178.stdout:The following additional packages will be installed: 2024-04-26T04:17:59.779 INFO:teuthology.orchestra.run.smithi178.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-26T04:17:59.779 INFO:teuthology.orchestra.run.smithi178.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-26T04:17:59.779 INFO:teuthology.orchestra.run.smithi178.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-26T04:17:59.780 INFO:teuthology.orchestra.run.smithi178.stdout:Suggested packages: 2024-04-26T04:17:59.780 INFO:teuthology.orchestra.run.smithi178.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-26T04:17:59.781 INFO:teuthology.orchestra.run.smithi178.stdout:Recommended packages: 2024-04-26T04:17:59.781 INFO:teuthology.orchestra.run.smithi178.stdout: thermald 2024-04-26T04:17:59.806 INFO:teuthology.orchestra.run.smithi137.stdout:The following packages were automatically installed and are no longer required: 2024-04-26T04:17:59.806 INFO:teuthology.orchestra.run.smithi137.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-26T04:17:59.807 INFO:teuthology.orchestra.run.smithi137.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-26T04:17:59.807 INFO:teuthology.orchestra.run.smithi137.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-26T04:17:59.807 INFO:teuthology.orchestra.run.smithi137.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-26T04:17:59.807 INFO:teuthology.orchestra.run.smithi137.stdout: python2.7-minimal 2024-04-26T04:17:59.807 INFO:teuthology.orchestra.run.smithi137.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-26T04:17:59.808 INFO:teuthology.orchestra.run.smithi137.stdout:The following additional packages will be installed: 2024-04-26T04:17:59.808 INFO:teuthology.orchestra.run.smithi137.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-26T04:17:59.809 INFO:teuthology.orchestra.run.smithi137.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-26T04:17:59.809 INFO:teuthology.orchestra.run.smithi137.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-26T04:17:59.810 INFO:teuthology.orchestra.run.smithi137.stdout:Suggested packages: 2024-04-26T04:17:59.810 INFO:teuthology.orchestra.run.smithi137.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-26T04:17:59.810 INFO:teuthology.orchestra.run.smithi137.stdout:Recommended packages: 2024-04-26T04:17:59.810 INFO:teuthology.orchestra.run.smithi137.stdout: thermald 2024-04-26T04:17:59.833 INFO:teuthology.orchestra.run.smithi178.stdout:The following NEW packages will be installed: 2024-04-26T04:17:59.833 INFO:teuthology.orchestra.run.smithi178.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-26T04:17:59.833 INFO:teuthology.orchestra.run.smithi178.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-26T04:17:59.833 INFO:teuthology.orchestra.run.smithi178.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-26T04:17:59.834 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages will be upgraded: 2024-04-26T04:17:59.835 INFO:teuthology.orchestra.run.smithi178.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-26T04:17:59.862 INFO:teuthology.orchestra.run.smithi137.stdout:The following NEW packages will be installed: 2024-04-26T04:17:59.862 INFO:teuthology.orchestra.run.smithi137.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-26T04:17:59.862 INFO:teuthology.orchestra.run.smithi137.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-26T04:17:59.863 INFO:teuthology.orchestra.run.smithi137.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-26T04:17:59.864 INFO:teuthology.orchestra.run.smithi137.stdout:The following packages will be upgraded: 2024-04-26T04:17:59.865 INFO:teuthology.orchestra.run.smithi137.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-26T04:17:59.892 INFO:teuthology.orchestra.run.smithi178.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-26T04:17:59.892 INFO:teuthology.orchestra.run.smithi178.stdout:Need to get 113 MB of archives. 2024-04-26T04:17:59.892 INFO:teuthology.orchestra.run.smithi178.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-26T04:17:59.892 INFO:teuthology.orchestra.run.smithi178.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-26T04:17:59.925 INFO:teuthology.orchestra.run.smithi137.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-26T04:17:59.925 INFO:teuthology.orchestra.run.smithi137.stdout:Need to get 113 MB of archives. 2024-04-26T04:17:59.925 INFO:teuthology.orchestra.run.smithi137.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-26T04:17:59.925 INFO:teuthology.orchestra.run.smithi137.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-26T04:18:00.352 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-04-26T04:18:00.368 DEBUG:teuthology.orchestra.run.smithi113:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-26T04:18:00.430 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-04-26T04:18:00.595 INFO:teuthology.orchestra.run.smithi178.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-26T04:18:00.630 INFO:teuthology.orchestra.run.smithi113.stdout:Building dependency tree... 2024-04-26T04:18:00.630 INFO:teuthology.orchestra.run.smithi113.stdout:Reading state information... 2024-04-26T04:18:00.704 INFO:teuthology.orchestra.run.smithi137.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-26T04:18:00.766 INFO:teuthology.orchestra.run.smithi178.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-26T04:18:00.803 INFO:teuthology.orchestra.run.smithi113.stdout:The following packages were automatically installed and are no longer required: 2024-04-26T04:18:00.803 INFO:teuthology.orchestra.run.smithi113.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-26T04:18:00.803 INFO:teuthology.orchestra.run.smithi113.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-26T04:18:00.803 INFO:teuthology.orchestra.run.smithi113.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-26T04:18:00.804 INFO:teuthology.orchestra.run.smithi113.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-26T04:18:00.804 INFO:teuthology.orchestra.run.smithi113.stdout: python2.7-minimal 2024-04-26T04:18:00.804 INFO:teuthology.orchestra.run.smithi113.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-26T04:18:00.805 INFO:teuthology.orchestra.run.smithi113.stdout:The following additional packages will be installed: 2024-04-26T04:18:00.805 INFO:teuthology.orchestra.run.smithi113.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-26T04:18:00.805 INFO:teuthology.orchestra.run.smithi113.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-26T04:18:00.806 INFO:teuthology.orchestra.run.smithi113.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-26T04:18:00.810 INFO:teuthology.orchestra.run.smithi113.stdout:Suggested packages: 2024-04-26T04:18:00.810 INFO:teuthology.orchestra.run.smithi113.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-26T04:18:00.810 INFO:teuthology.orchestra.run.smithi113.stdout:Recommended packages: 2024-04-26T04:18:00.810 INFO:teuthology.orchestra.run.smithi113.stdout: thermald 2024-04-26T04:18:00.821 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-04-26T04:18:00.838 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-26T04:18:00.861 INFO:teuthology.orchestra.run.smithi113.stdout:The following NEW packages will be installed: 2024-04-26T04:18:00.861 INFO:teuthology.orchestra.run.smithi113.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-26T04:18:00.861 INFO:teuthology.orchestra.run.smithi113.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-26T04:18:00.862 INFO:teuthology.orchestra.run.smithi113.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-26T04:18:00.863 INFO:teuthology.orchestra.run.smithi113.stdout:The following packages will be upgraded: 2024-04-26T04:18:00.863 INFO:teuthology.orchestra.run.smithi113.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-26T04:18:00.901 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-04-26T04:18:00.907 INFO:teuthology.orchestra.run.smithi137.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-26T04:18:01.043 INFO:teuthology.orchestra.run.smithi113.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-26T04:18:01.043 INFO:teuthology.orchestra.run.smithi113.stdout:Need to get 113 MB of archives. 2024-04-26T04:18:01.043 INFO:teuthology.orchestra.run.smithi113.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-26T04:18:01.043 INFO:teuthology.orchestra.run.smithi113.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-26T04:18:01.108 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-04-26T04:18:01.109 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-04-26T04:18:01.292 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2024-04-26T04:18:01.292 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-26T04:18:01.292 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-26T04:18:01.293 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-26T04:18:01.293 INFO:teuthology.orchestra.run.smithi012.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-26T04:18:01.293 INFO:teuthology.orchestra.run.smithi012.stdout: python2.7-minimal 2024-04-26T04:18:01.293 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-26T04:18:01.294 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2024-04-26T04:18:01.294 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-26T04:18:01.294 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-26T04:18:01.294 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-26T04:18:01.296 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2024-04-26T04:18:01.296 INFO:teuthology.orchestra.run.smithi012.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-26T04:18:01.296 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2024-04-26T04:18:01.296 INFO:teuthology.orchestra.run.smithi012.stdout: thermald 2024-04-26T04:18:01.346 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2024-04-26T04:18:01.346 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-26T04:18:01.346 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-26T04:18:01.347 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-26T04:18:01.348 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages will be upgraded: 2024-04-26T04:18:01.349 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-26T04:18:01.479 INFO:teuthology.orchestra.run.smithi012.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-26T04:18:01.479 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 113 MB of archives. 2024-04-26T04:18:01.479 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-26T04:18:01.479 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-26T04:18:02.525 INFO:teuthology.orchestra.run.smithi113.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-26T04:18:02.680 INFO:teuthology.orchestra.run.smithi137.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-26T04:18:02.680 INFO:teuthology.orchestra.run.smithi137.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-26T04:18:02.680 INFO:teuthology.orchestra.run.smithi137.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-26T04:18:02.777 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-26T04:18:02.871 INFO:teuthology.orchestra.run.smithi178.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-26T04:18:02.872 INFO:teuthology.orchestra.run.smithi178.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-26T04:18:02.872 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-26T04:18:02.878 INFO:teuthology.orchestra.run.smithi113.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-26T04:18:02.938 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-26T04:18:03.300 INFO:teuthology.orchestra.run.smithi178.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-26T04:18:03.373 INFO:teuthology.orchestra.run.smithi137.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-26T04:18:03.473 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-26T04:18:03.548 INFO:teuthology.orchestra.run.smithi137.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-26T04:18:03.912 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 113 MB in 4s (31.2 MB/s) 2024-04-26T04:18:03.952 INFO:teuthology.orchestra.run.smithi137.stdout:Fetched 113 MB in 4s (30.8 MB/s) 2024-04-26T04:18:04.094 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-26T04:18:04.123 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-26T04:18:04.322 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-26T04:18:04.322 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-26T04:18:04.323 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-26T04:18:04.592 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-26T04:18:04.628 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-26T04:18:05.045 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 113 MB in 3s (34.6 MB/s) 2024-04-26T04:18:05.130 INFO:teuthology.orchestra.run.smithi113.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-26T04:18:05.130 INFO:teuthology.orchestra.run.smithi113.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-26T04:18:05.133 INFO:teuthology.orchestra.run.smithi113.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-26T04:18:05.202 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-26T04:18:05.648 INFO:teuthology.orchestra.run.smithi113.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-26T04:18:05.741 INFO:teuthology.orchestra.run.smithi113.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-26T04:18:06.120 INFO:teuthology.orchestra.run.smithi137.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-26T04:18:06.124 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:06.130 INFO:teuthology.orchestra.run.smithi113.stdout:Fetched 113 MB in 5s (23.3 MB/s) 2024-04-26T04:18:06.149 INFO:teuthology.orchestra.run.smithi178.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-26T04:18:06.153 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:06.204 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:06.214 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:06.304 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-26T04:18:07.254 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-26T04:18:07.258 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:07.367 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:08.321 INFO:teuthology.orchestra.run.smithi113.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-26T04:18:08.325 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:08.400 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:08.477 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-26T04:18:08.499 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:08.712 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:08.882 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-26T04:18:08.903 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:08.993 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:09.174 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-26T04:18:09.195 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:09.257 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:09.427 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-26T04:18:09.448 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:09.477 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-26T04:18:09.498 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:09.535 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:09.554 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:10.030 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-26T04:18:10.051 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:10.088 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:10.935 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-26T04:18:10.956 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:11.045 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:11.497 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-26T04:18:11.517 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:11.555 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:15.575 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-26T04:18:15.691 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-26T04:18:15.999 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-26T04:18:16.035 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-26T04:18:16.095 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-26T04:18:16.160 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-26T04:18:16.423 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-26T04:18:16.445 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-26T04:18:16.459 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-26T04:18:16.489 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-26T04:18:16.563 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-26T04:18:16.875 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-26T04:18:16.883 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-26T04:18:16.897 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-26T04:18:16.966 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-26T04:18:17.007 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-26T04:18:17.315 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-26T04:18:17.444 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-26T04:18:17.628 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-26T04:18:17.744 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-26T04:18:17.755 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-26T04:18:17.778 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-26T04:18:17.822 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-26T04:18:18.043 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-26T04:18:18.164 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-26T04:18:18.459 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-26T04:18:18.481 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-26T04:18:18.525 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-26T04:18:21.898 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-26T04:18:21.935 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:21.964 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:22.377 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-26T04:18:22.415 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:22.460 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:23.334 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-26T04:18:23.371 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:23.400 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:23.768 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-26T04:18:23.788 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-26T04:18:23.826 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:25.057 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-26T04:18:25.158 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-26T04:18:25.483 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-26T04:18:25.604 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:25.658 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-26T04:18:25.730 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-26T04:18:25.783 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-26T04:18:25.867 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:26.056 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-26T04:18:26.117 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-26T04:18:26.189 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-26T04:18:26.238 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:26.364 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-26T04:18:26.501 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:26.506 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-26T04:18:26.634 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:26.735 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-26T04:18:26.838 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:26.989 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-26T04:18:27.245 INFO:teuthology.orchestra.run.smithi137.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-26T04:18:27.245 INFO:teuthology.orchestra.run.smithi137.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-26T04:18:27.245 INFO:teuthology.orchestra.run.smithi137.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-26T04:18:27.245 INFO:teuthology.orchestra.run.smithi137.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:27.727 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:27.827 INFO:teuthology.orchestra.run.smithi178.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-26T04:18:27.828 INFO:teuthology.orchestra.run.smithi178.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-26T04:18:27.828 INFO:teuthology.orchestra.run.smithi178.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-26T04:18:27.828 INFO:teuthology.orchestra.run.smithi178.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:27.832 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-26T04:18:27.996 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:28.140 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-26T04:18:28.262 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:28.371 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-26T04:18:28.500 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:28.829 INFO:teuthology.orchestra.run.smithi012.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-26T04:18:28.829 INFO:teuthology.orchestra.run.smithi012.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-26T04:18:28.829 INFO:teuthology.orchestra.run.smithi012.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-26T04:18:28.829 INFO:teuthology.orchestra.run.smithi012.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:28.990 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:29.105 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:29.373 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:29.819 INFO:teuthology.orchestra.run.smithi113.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-26T04:18:29.819 INFO:teuthology.orchestra.run.smithi113.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-26T04:18:29.819 INFO:teuthology.orchestra.run.smithi113.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-26T04:18:29.819 INFO:teuthology.orchestra.run.smithi113.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:29.977 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:30.451 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:30.513 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-26T04:18:30.639 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-26T04:18:30.774 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:30.774 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-26T04:18:30.831 INFO:teuthology.orchestra.run.smithi137.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-26T04:18:30.832 INFO:teuthology.orchestra.run.smithi137.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:30.894 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-26T04:18:31.018 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:31.075 INFO:teuthology.orchestra.run.smithi178.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-26T04:18:31.075 INFO:teuthology.orchestra.run.smithi178.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:31.488 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:31.868 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-26T04:18:31.998 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-26T04:18:32.629 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:32.697 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-26T04:18:32.697 INFO:teuthology.orchestra.run.smithi012.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:32.882 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-26T04:18:33.700 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-26T04:18:33.801 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-26T04:18:33.850 INFO:teuthology.orchestra.run.smithi113.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-26T04:18:33.850 INFO:teuthology.orchestra.run.smithi113.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:44.881 INFO:teuthology.orchestra.run.smithi137.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-26T04:18:44.881 INFO:teuthology.orchestra.run.smithi137.stdout:Sourcing file `/etc/default/grub' 2024-04-26T04:18:44.890 INFO:teuthology.orchestra.run.smithi137.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-26T04:18:44.903 INFO:teuthology.orchestra.run.smithi137.stdout:Generating grub configuration file ... 2024-04-26T04:18:45.243 INFO:teuthology.orchestra.run.smithi137.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-26T04:18:45.252 INFO:teuthology.orchestra.run.smithi178.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-26T04:18:45.252 INFO:teuthology.orchestra.run.smithi178.stdout:Sourcing file `/etc/default/grub' 2024-04-26T04:18:45.262 INFO:teuthology.orchestra.run.smithi178.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-26T04:18:45.270 INFO:teuthology.orchestra.run.smithi137.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:45.290 INFO:teuthology.orchestra.run.smithi178.stdout:Generating grub configuration file ... 2024-04-26T04:18:45.564 INFO:teuthology.orchestra.run.smithi178.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-26T04:18:45.582 INFO:teuthology.orchestra.run.smithi178.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:45.651 INFO:teuthology.orchestra.run.smithi137.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-26T04:18:45.657 INFO:teuthology.orchestra.run.smithi137.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-26T04:18:45.722 INFO:teuthology.orchestra.run.smithi137.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-26T04:18:45.727 INFO:teuthology.orchestra.run.smithi137.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-26T04:18:45.872 INFO:teuthology.orchestra.run.smithi178.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-26T04:18:45.878 INFO:teuthology.orchestra.run.smithi178.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-26T04:18:45.886 INFO:teuthology.orchestra.run.smithi137.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-26T04:18:45.886 INFO:teuthology.orchestra.run.smithi137.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-26T04:18:45.887 INFO:teuthology.orchestra.run.smithi137.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-26T04:18:45.936 INFO:teuthology.orchestra.run.smithi137.stdout:done 2024-04-26T04:18:45.946 INFO:teuthology.orchestra.run.smithi178.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-26T04:18:45.951 INFO:teuthology.orchestra.run.smithi178.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-26T04:18:46.101 INFO:teuthology.orchestra.run.smithi178.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-26T04:18:46.102 INFO:teuthology.orchestra.run.smithi178.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-26T04:18:46.102 INFO:teuthology.orchestra.run.smithi178.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-26T04:18:46.146 INFO:teuthology.orchestra.run.smithi178.stdout:done 2024-04-26T04:18:46.408 DEBUG:teuthology.orchestra.run.smithi137:> dpkg -s linux-image-generic 2024-04-26T04:18:46.428 INFO:teuthology.orchestra.run.smithi137.stdout:Package: linux-image-generic 2024-04-26T04:18:46.428 INFO:teuthology.orchestra.run.smithi137.stdout:Status: install ok installed 2024-04-26T04:18:46.428 INFO:teuthology.orchestra.run.smithi137.stdout:Priority: optional 2024-04-26T04:18:46.428 INFO:teuthology.orchestra.run.smithi137.stdout:Section: kernel 2024-04-26T04:18:46.429 INFO:teuthology.orchestra.run.smithi137.stdout:Installed-Size: 21 2024-04-26T04:18:46.429 INFO:teuthology.orchestra.run.smithi137.stdout:Maintainer: Ubuntu Kernel Team 2024-04-26T04:18:46.429 INFO:teuthology.orchestra.run.smithi137.stdout:Architecture: amd64 2024-04-26T04:18:46.429 INFO:teuthology.orchestra.run.smithi137.stdout:Source: linux-meta 2024-04-26T04:18:46.429 INFO:teuthology.orchestra.run.smithi137.stdout:Version: 5.15.0.105.102 2024-04-26T04:18:46.429 INFO:teuthology.orchestra.run.smithi137.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-26T04:18:46.429 INFO:teuthology.orchestra.run.smithi137.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-26T04:18:46.429 INFO:teuthology.orchestra.run.smithi137.stdout:Recommends: thermald 2024-04-26T04:18:46.429 INFO:teuthology.orchestra.run.smithi137.stdout:Description: Generic Linux kernel image 2024-04-26T04:18:46.429 INFO:teuthology.orchestra.run.smithi137.stdout: This package will always depend on the latest generic kernel image 2024-04-26T04:18:46.429 INFO:teuthology.orchestra.run.smithi137.stdout: available. 2024-04-26T04:18:46.430 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-26T04:18:46.430 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-26T04:18:46.430 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-26T04:18:46.430 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-04-26T04:18:46.430 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi137.front.sepia.ceph.com, path=None, version=distro) 2024-04-26T04:18:46.430 DEBUG:teuthology.orchestra.run.smithi137:> sudo apt-get clean 2024-04-26T04:18:46.633 DEBUG:teuthology.orchestra.run.smithi178:> dpkg -s linux-image-generic 2024-04-26T04:18:46.659 INFO:teuthology.orchestra.run.smithi178.stdout:Package: linux-image-generic 2024-04-26T04:18:46.660 INFO:teuthology.orchestra.run.smithi178.stdout:Status: install ok installed 2024-04-26T04:18:46.660 INFO:teuthology.orchestra.run.smithi178.stdout:Priority: optional 2024-04-26T04:18:46.660 INFO:teuthology.orchestra.run.smithi178.stdout:Section: kernel 2024-04-26T04:18:46.660 INFO:teuthology.orchestra.run.smithi178.stdout:Installed-Size: 21 2024-04-26T04:18:46.660 INFO:teuthology.orchestra.run.smithi178.stdout:Maintainer: Ubuntu Kernel Team 2024-04-26T04:18:46.660 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: amd64 2024-04-26T04:18:46.660 INFO:teuthology.orchestra.run.smithi178.stdout:Source: linux-meta 2024-04-26T04:18:46.660 INFO:teuthology.orchestra.run.smithi178.stdout:Version: 5.15.0.105.102 2024-04-26T04:18:46.660 INFO:teuthology.orchestra.run.smithi178.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-26T04:18:46.660 INFO:teuthology.orchestra.run.smithi178.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-26T04:18:46.660 INFO:teuthology.orchestra.run.smithi178.stdout:Recommends: thermald 2024-04-26T04:18:46.660 INFO:teuthology.orchestra.run.smithi178.stdout:Description: Generic Linux kernel image 2024-04-26T04:18:46.661 INFO:teuthology.orchestra.run.smithi178.stdout: This package will always depend on the latest generic kernel image 2024-04-26T04:18:46.661 INFO:teuthology.orchestra.run.smithi178.stdout: available. 2024-04-26T04:18:46.661 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-26T04:18:46.661 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-26T04:18:46.661 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-26T04:18:46.661 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-04-26T04:18:46.661 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi178.front.sepia.ceph.com, path=None, version=distro) 2024-04-26T04:18:46.661 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get clean 2024-04-26T04:18:46.664 DEBUG:teuthology.orchestra.run.smithi137:> sudo apt-get update 2024-04-26T04:18:46.858 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-26T04:18:46.867 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get update 2024-04-26T04:18:46.905 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-26T04:18:46.987 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-26T04:18:47.034 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-26T04:18:47.034 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-26T04:18:47.068 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-26T04:18:47.069 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-26T04:18:47.107 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-26T04:18:47.272 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-26T04:18:47.272 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub' 2024-04-26T04:18:47.276 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-26T04:18:47.314 INFO:teuthology.orchestra.run.smithi012.stdout:Generating grub configuration file ... 2024-04-26T04:18:47.608 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-26T04:18:47.628 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:47.665 INFO:teuthology.orchestra.run.smithi113.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-26T04:18:47.665 INFO:teuthology.orchestra.run.smithi113.stdout:Sourcing file `/etc/default/grub' 2024-04-26T04:18:47.676 INFO:teuthology.orchestra.run.smithi113.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-26T04:18:47.700 INFO:teuthology.orchestra.run.smithi113.stdout:Generating grub configuration file ... 2024-04-26T04:18:47.933 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-26T04:18:47.939 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-26T04:18:47.943 INFO:teuthology.orchestra.run.smithi113.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-26T04:18:47.969 INFO:teuthology.orchestra.run.smithi113.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:48.013 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-26T04:18:48.018 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-26T04:18:48.208 INFO:teuthology.orchestra.run.smithi012.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-26T04:18:48.208 INFO:teuthology.orchestra.run.smithi012.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-26T04:18:48.208 INFO:teuthology.orchestra.run.smithi012.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-26T04:18:48.215 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-04-26T04:18:48.233 DEBUG:teuthology.orchestra.run.smithi137:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-26T04:18:48.241 INFO:teuthology.orchestra.run.smithi113.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-26T04:18:48.247 INFO:teuthology.orchestra.run.smithi113.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-26T04:18:48.248 INFO:teuthology.orchestra.run.smithi012.stdout:done 2024-04-26T04:18:48.292 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-04-26T04:18:48.314 INFO:teuthology.orchestra.run.smithi113.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-26T04:18:48.318 INFO:teuthology.orchestra.run.smithi113.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-26T04:18:48.365 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-04-26T04:18:48.382 DEBUG:teuthology.orchestra.run.smithi178:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-26T04:18:48.435 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-04-26T04:18:48.456 INFO:teuthology.orchestra.run.smithi113.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-26T04:18:48.456 INFO:teuthology.orchestra.run.smithi113.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-26T04:18:48.456 INFO:teuthology.orchestra.run.smithi113.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-26T04:18:48.490 INFO:teuthology.orchestra.run.smithi137.stdout:Building dependency tree... 2024-04-26T04:18:48.491 INFO:teuthology.orchestra.run.smithi137.stdout:Reading state information... 2024-04-26T04:18:48.493 INFO:teuthology.orchestra.run.smithi113.stdout:done 2024-04-26T04:18:48.612 INFO:teuthology.orchestra.run.smithi178.stdout:Building dependency tree... 2024-04-26T04:18:48.613 INFO:teuthology.orchestra.run.smithi178.stdout:Reading state information... 2024-04-26T04:18:48.659 INFO:teuthology.orchestra.run.smithi137.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-26T04:18:48.659 INFO:teuthology.orchestra.run.smithi137.stdout:The following packages were automatically installed and are no longer required: 2024-04-26T04:18:48.660 INFO:teuthology.orchestra.run.smithi137.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-26T04:18:48.660 INFO:teuthology.orchestra.run.smithi137.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-26T04:18:48.660 INFO:teuthology.orchestra.run.smithi137.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-26T04:18:48.660 INFO:teuthology.orchestra.run.smithi137.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-26T04:18:48.660 INFO:teuthology.orchestra.run.smithi137.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-26T04:18:48.660 INFO:teuthology.orchestra.run.smithi137.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-26T04:18:48.698 INFO:teuthology.orchestra.run.smithi137.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-26T04:18:48.700 DEBUG:teuthology.orchestra.run.smithi137:> dpkg -s linux-image-generic 2024-04-26T04:18:48.716 INFO:teuthology.orchestra.run.smithi137.stdout:Package: linux-image-generic 2024-04-26T04:18:48.716 INFO:teuthology.orchestra.run.smithi137.stdout:Status: install ok installed 2024-04-26T04:18:48.716 INFO:teuthology.orchestra.run.smithi137.stdout:Priority: optional 2024-04-26T04:18:48.716 INFO:teuthology.orchestra.run.smithi137.stdout:Section: kernel 2024-04-26T04:18:48.716 INFO:teuthology.orchestra.run.smithi137.stdout:Installed-Size: 21 2024-04-26T04:18:48.716 INFO:teuthology.orchestra.run.smithi137.stdout:Maintainer: Ubuntu Kernel Team 2024-04-26T04:18:48.716 INFO:teuthology.orchestra.run.smithi137.stdout:Architecture: amd64 2024-04-26T04:18:48.716 INFO:teuthology.orchestra.run.smithi137.stdout:Source: linux-meta 2024-04-26T04:18:48.716 INFO:teuthology.orchestra.run.smithi137.stdout:Version: 5.15.0.105.102 2024-04-26T04:18:48.716 INFO:teuthology.orchestra.run.smithi137.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-26T04:18:48.717 INFO:teuthology.orchestra.run.smithi137.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-26T04:18:48.717 INFO:teuthology.orchestra.run.smithi137.stdout:Recommends: thermald 2024-04-26T04:18:48.717 INFO:teuthology.orchestra.run.smithi137.stdout:Description: Generic Linux kernel image 2024-04-26T04:18:48.717 INFO:teuthology.orchestra.run.smithi137.stdout: This package will always depend on the latest generic kernel image 2024-04-26T04:18:48.717 INFO:teuthology.orchestra.run.smithi137.stdout: available. 2024-04-26T04:18:48.717 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-04-26T04:18:48.745 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2024-04-26T04:18:48.763 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.8v9PjautWA 2024-04-26T04:18:48.763 DEBUG:teuthology.orchestra.run.smithi137:> sudo cp /boot/grub/grub.cfg /tmp/tmp.8v9PjautWA 2024-04-26T04:18:48.764 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2024-04-26T04:18:48.764 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2024-04-26T04:18:48.764 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2024-04-26T04:18:48.764 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2024-04-26T04:18:48.764 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 21 2024-04-26T04:18:48.765 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2024-04-26T04:18:48.765 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2024-04-26T04:18:48.765 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2024-04-26T04:18:48.765 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.15.0.105.102 2024-04-26T04:18:48.765 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-26T04:18:48.765 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-26T04:18:48.765 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2024-04-26T04:18:48.765 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2024-04-26T04:18:48.765 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2024-04-26T04:18:48.765 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2024-04-26T04:18:48.765 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-26T04:18:48.765 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-26T04:18:48.765 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-26T04:18:48.766 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-26T04:18:48.766 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=None, version=distro) 2024-04-26T04:18:48.766 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2024-04-26T04:18:48.769 INFO:teuthology.orchestra.run.smithi178.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-26T04:18:48.769 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages were automatically installed and are no longer required: 2024-04-26T04:18:48.770 INFO:teuthology.orchestra.run.smithi178.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-26T04:18:48.770 INFO:teuthology.orchestra.run.smithi178.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-26T04:18:48.770 INFO:teuthology.orchestra.run.smithi178.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-26T04:18:48.770 INFO:teuthology.orchestra.run.smithi178.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-26T04:18:48.770 INFO:teuthology.orchestra.run.smithi178.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-26T04:18:48.770 INFO:teuthology.orchestra.run.smithi178.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-26T04:18:48.806 INFO:teuthology.orchestra.run.smithi178.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-26T04:18:48.807 DEBUG:teuthology.orchestra.run.smithi178:> dpkg -s linux-image-generic 2024-04-26T04:18:48.822 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.8v9PjautWA 2024-04-26T04:18:48.866 INFO:teuthology.orchestra.run.smithi178.stdout:Package: linux-image-generic 2024-04-26T04:18:48.866 INFO:teuthology.orchestra.run.smithi178.stdout:Status: install ok installed 2024-04-26T04:18:48.866 INFO:teuthology.orchestra.run.smithi178.stdout:Priority: optional 2024-04-26T04:18:48.867 INFO:teuthology.orchestra.run.smithi178.stdout:Section: kernel 2024-04-26T04:18:48.867 INFO:teuthology.orchestra.run.smithi178.stdout:Installed-Size: 21 2024-04-26T04:18:48.867 INFO:teuthology.orchestra.run.smithi178.stdout:Maintainer: Ubuntu Kernel Team 2024-04-26T04:18:48.867 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: amd64 2024-04-26T04:18:48.867 INFO:teuthology.orchestra.run.smithi178.stdout:Source: linux-meta 2024-04-26T04:18:48.867 INFO:teuthology.orchestra.run.smithi178.stdout:Version: 5.15.0.105.102 2024-04-26T04:18:48.868 INFO:teuthology.orchestra.run.smithi178.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-26T04:18:48.868 INFO:teuthology.orchestra.run.smithi178.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-26T04:18:48.868 INFO:teuthology.orchestra.run.smithi178.stdout:Recommends: thermald 2024-04-26T04:18:48.868 INFO:teuthology.orchestra.run.smithi178.stdout:Description: Generic Linux kernel image 2024-04-26T04:18:48.868 INFO:teuthology.orchestra.run.smithi178.stdout: This package will always depend on the latest generic kernel image 2024-04-26T04:18:48.868 INFO:teuthology.orchestra.run.smithi178.stdout: available. 2024-04-26T04:18:48.869 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-04-26T04:18:48.915 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.yLPH44wgCH 2024-04-26T04:18:48.915 DEBUG:teuthology.orchestra.run.smithi178:> sudo cp /boot/grub/grub.cfg /tmp/tmp.yLPH44wgCH 2024-04-26T04:18:48.951 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.8v9PjautWA is 10KB 2024-04-26T04:18:48.957 DEBUG:teuthology.orchestra.run.smithi113:> dpkg -s linux-image-generic 2024-04-26T04:18:48.977 INFO:teuthology.orchestra.run.smithi113.stdout:Package: linux-image-generic 2024-04-26T04:18:48.977 INFO:teuthology.orchestra.run.smithi113.stdout:Status: install ok installed 2024-04-26T04:18:48.977 INFO:teuthology.orchestra.run.smithi113.stdout:Priority: optional 2024-04-26T04:18:48.978 INFO:teuthology.orchestra.run.smithi113.stdout:Section: kernel 2024-04-26T04:18:48.978 INFO:teuthology.orchestra.run.smithi113.stdout:Installed-Size: 21 2024-04-26T04:18:48.978 INFO:teuthology.orchestra.run.smithi113.stdout:Maintainer: Ubuntu Kernel Team 2024-04-26T04:18:48.978 INFO:teuthology.orchestra.run.smithi113.stdout:Architecture: amd64 2024-04-26T04:18:48.978 INFO:teuthology.orchestra.run.smithi113.stdout:Source: linux-meta 2024-04-26T04:18:48.978 INFO:teuthology.orchestra.run.smithi113.stdout:Version: 5.15.0.105.102 2024-04-26T04:18:48.978 INFO:teuthology.orchestra.run.smithi113.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-26T04:18:48.978 INFO:teuthology.orchestra.run.smithi113.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-26T04:18:48.978 INFO:teuthology.orchestra.run.smithi113.stdout:Recommends: thermald 2024-04-26T04:18:48.978 INFO:teuthology.orchestra.run.smithi113.stdout:Description: Generic Linux kernel image 2024-04-26T04:18:48.978 INFO:teuthology.orchestra.run.smithi113.stdout: This package will always depend on the latest generic kernel image 2024-04-26T04:18:48.978 INFO:teuthology.orchestra.run.smithi113.stdout: available. 2024-04-26T04:18:48.979 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-26T04:18:48.979 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-26T04:18:48.980 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-26T04:18:48.980 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-26T04:18:48.980 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi113.front.sepia.ceph.com, path=None, version=distro) 2024-04-26T04:18:48.980 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-get clean 2024-04-26T04:18:48.982 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.yLPH44wgCH 2024-04-26T04:18:48.989 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-04-26T04:18:49.007 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.8v9PjautWA 2024-04-26T04:18:49.012 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-26T04:18:49.066 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:18:49.066 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-26T04:18:49.066 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-26T04:18:49.090 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.yLPH44wgCH is 10KB 2024-04-26T04:18:49.103 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.yLPH44wgCH 2024-04-26T04:18:49.109 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-26T04:18:49.120 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-get update 2024-04-26T04:18:49.142 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-26T04:18:49.143 DEBUG:teuthology.orchestra.run.smithi137:> sudo update-grub 2024-04-26T04:18:49.163 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:18:49.163 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-26T04:18:49.163 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-26T04:18:49.241 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-26T04:18:49.242 DEBUG:teuthology.orchestra.run.smithi178:> sudo update-grub 2024-04-26T04:18:49.249 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-26T04:18:49.261 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-26T04:18:49.286 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-26T04:18:49.314 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-26T04:18:49.340 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-26T04:18:49.369 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-26T04:18:49.439 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-26T04:18:49.515 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-26T04:18:50.473 INFO:teuthology.orchestra.run.smithi178.stderr:Sourcing file `/etc/default/grub' 2024-04-26T04:18:50.474 INFO:teuthology.orchestra.run.smithi178.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-26T04:18:50.476 INFO:teuthology.orchestra.run.smithi178.stderr:Generating grub configuration file ... 2024-04-26T04:18:50.495 INFO:teuthology.orchestra.run.smithi137.stderr:Sourcing file `/etc/default/grub' 2024-04-26T04:18:50.496 INFO:teuthology.orchestra.run.smithi137.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-26T04:18:50.505 INFO:teuthology.orchestra.run.smithi137.stderr:Generating grub configuration file ... 2024-04-26T04:18:50.596 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-04-26T04:18:50.614 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-26T04:18:50.636 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-04-26T04:18:50.650 INFO:teuthology.orchestra.run.smithi178.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-26T04:18:50.655 DEBUG:teuthology.orchestra.run.smithi113:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-26T04:18:50.657 INFO:teuthology.orchestra.run.smithi178.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:50.678 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-04-26T04:18:50.706 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-04-26T04:18:50.763 INFO:teuthology.orchestra.run.smithi137.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-26T04:18:50.769 INFO:teuthology.orchestra.run.smithi137.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:50.882 INFO:teuthology.orchestra.run.smithi113.stdout:Building dependency tree... 2024-04-26T04:18:50.883 INFO:teuthology.orchestra.run.smithi178.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-26T04:18:50.883 INFO:teuthology.orchestra.run.smithi113.stdout:Reading state information... 2024-04-26T04:18:50.883 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-04-26T04:18:50.883 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-04-26T04:18:50.888 INFO:teuthology.orchestra.run.smithi178.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-26T04:18:50.932 INFO:teuthology.orchestra.run.smithi178.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-26T04:18:50.938 INFO:teuthology.orchestra.run.smithi178.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-26T04:18:51.038 INFO:teuthology.orchestra.run.smithi113.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-26T04:18:51.038 INFO:teuthology.orchestra.run.smithi113.stdout:The following packages were automatically installed and are no longer required: 2024-04-26T04:18:51.038 INFO:teuthology.orchestra.run.smithi113.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-26T04:18:51.038 INFO:teuthology.orchestra.run.smithi113.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-26T04:18:51.038 INFO:teuthology.orchestra.run.smithi113.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-26T04:18:51.038 INFO:teuthology.orchestra.run.smithi113.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-26T04:18:51.039 INFO:teuthology.orchestra.run.smithi113.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-26T04:18:51.039 INFO:teuthology.orchestra.run.smithi113.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-26T04:18:51.064 INFO:teuthology.orchestra.run.smithi137.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-26T04:18:51.065 INFO:teuthology.orchestra.run.smithi012.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-26T04:18:51.065 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2024-04-26T04:18:51.065 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-26T04:18:51.066 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-26T04:18:51.066 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-26T04:18:51.066 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-26T04:18:51.066 INFO:teuthology.orchestra.run.smithi012.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-26T04:18:51.067 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-26T04:18:51.067 INFO:teuthology.orchestra.run.smithi178.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-26T04:18:51.067 INFO:teuthology.orchestra.run.smithi178.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-26T04:18:51.067 INFO:teuthology.orchestra.run.smithi178.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-26T04:18:51.069 INFO:teuthology.orchestra.run.smithi137.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-26T04:18:51.077 INFO:teuthology.orchestra.run.smithi178.stderr:done 2024-04-26T04:18:51.078 INFO:teuthology.orchestra.run.smithi113.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-26T04:18:51.080 DEBUG:teuthology.orchestra.run.smithi113:> dpkg -s linux-image-generic 2024-04-26T04:18:51.101 DEBUG:teuthology.orchestra.run.smithi178:> sudo shutdown -r now 2024-04-26T04:18:51.108 INFO:teuthology.orchestra.run.smithi012.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-26T04:18:51.110 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2024-04-26T04:18:51.113 INFO:teuthology.orchestra.run.smithi137.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-26T04:18:51.118 INFO:teuthology.orchestra.run.smithi137.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-26T04:18:51.127 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2024-04-26T04:18:51.127 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2024-04-26T04:18:51.127 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2024-04-26T04:18:51.127 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2024-04-26T04:18:51.127 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 21 2024-04-26T04:18:51.127 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2024-04-26T04:18:51.127 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2024-04-26T04:18:51.127 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2024-04-26T04:18:51.127 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.15.0.105.102 2024-04-26T04:18:51.127 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-26T04:18:51.127 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-26T04:18:51.127 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2024-04-26T04:18:51.128 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2024-04-26T04:18:51.128 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2024-04-26T04:18:51.128 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2024-04-26T04:18:51.128 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-04-26T04:18:51.135 INFO:teuthology.orchestra.run.smithi113.stdout:Package: linux-image-generic 2024-04-26T04:18:51.135 INFO:teuthology.orchestra.run.smithi113.stdout:Status: install ok installed 2024-04-26T04:18:51.135 INFO:teuthology.orchestra.run.smithi113.stdout:Priority: optional 2024-04-26T04:18:51.135 INFO:teuthology.orchestra.run.smithi113.stdout:Section: kernel 2024-04-26T04:18:51.136 INFO:teuthology.orchestra.run.smithi113.stdout:Installed-Size: 21 2024-04-26T04:18:51.136 INFO:teuthology.orchestra.run.smithi113.stdout:Maintainer: Ubuntu Kernel Team 2024-04-26T04:18:51.136 INFO:teuthology.orchestra.run.smithi113.stdout:Architecture: amd64 2024-04-26T04:18:51.136 INFO:teuthology.orchestra.run.smithi113.stdout:Source: linux-meta 2024-04-26T04:18:51.136 INFO:teuthology.orchestra.run.smithi113.stdout:Version: 5.15.0.105.102 2024-04-26T04:18:51.136 INFO:teuthology.orchestra.run.smithi113.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-26T04:18:51.136 INFO:teuthology.orchestra.run.smithi113.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-26T04:18:51.136 INFO:teuthology.orchestra.run.smithi113.stdout:Recommends: thermald 2024-04-26T04:18:51.136 INFO:teuthology.orchestra.run.smithi113.stdout:Description: Generic Linux kernel image 2024-04-26T04:18:51.136 INFO:teuthology.orchestra.run.smithi113.stdout: This package will always depend on the latest generic kernel image 2024-04-26T04:18:51.136 INFO:teuthology.orchestra.run.smithi113.stdout: available. 2024-04-26T04:18:51.137 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-04-26T04:18:51.178 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.5mAjYlXd7L 2024-04-26T04:18:51.178 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub/grub.cfg /tmp/tmp.5mAjYlXd7L 2024-04-26T04:18:51.181 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.JZMfCaP5jO 2024-04-26T04:18:51.181 DEBUG:teuthology.orchestra.run.smithi113:> sudo cp /boot/grub/grub.cfg /tmp/tmp.JZMfCaP5jO 2024-04-26T04:18:51.234 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.5mAjYlXd7L 2024-04-26T04:18:51.235 INFO:teuthology.orchestra.run.smithi137.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-26T04:18:51.235 INFO:teuthology.orchestra.run.smithi137.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-26T04:18:51.235 INFO:teuthology.orchestra.run.smithi137.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-26T04:18:51.237 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.JZMfCaP5jO 2024-04-26T04:18:51.249 INFO:teuthology.orchestra.run.smithi137.stderr:done 2024-04-26T04:18:51.263 DEBUG:teuthology.orchestra.run.smithi137:> sudo shutdown -r now 2024-04-26T04:18:51.354 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.JZMfCaP5jO is 10KB 2024-04-26T04:18:51.360 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.5mAjYlXd7L is 10KB 2024-04-26T04:18:51.376 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.JZMfCaP5jO 2024-04-26T04:18:51.377 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.5mAjYlXd7L 2024-04-26T04:18:51.383 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-26T04:18:51.384 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-26T04:18:51.437 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:18:51.437 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-26T04:18:51.437 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-26T04:18:51.438 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:18:51.438 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-26T04:18:51.438 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-26T04:18:51.505 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-26T04:18:51.505 DEBUG:teuthology.orchestra.run.smithi113:> sudo update-grub 2024-04-26T04:18:51.514 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-26T04:18:51.514 DEBUG:teuthology.orchestra.run.smithi012:> sudo update-grub 2024-04-26T04:18:52.636 INFO:teuthology.orchestra.run.smithi113.stderr:Sourcing file `/etc/default/grub' 2024-04-26T04:18:52.637 INFO:teuthology.orchestra.run.smithi113.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-26T04:18:52.639 INFO:teuthology.orchestra.run.smithi113.stderr:Generating grub configuration file ... 2024-04-26T04:18:52.643 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub' 2024-04-26T04:18:52.644 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-26T04:18:52.646 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2024-04-26T04:18:52.796 INFO:teuthology.orchestra.run.smithi113.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-26T04:18:52.803 INFO:teuthology.orchestra.run.smithi113.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:52.965 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-26T04:18:52.972 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-26T04:18:53.026 INFO:teuthology.orchestra.run.smithi113.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-26T04:18:53.032 INFO:teuthology.orchestra.run.smithi113.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-26T04:18:53.073 INFO:teuthology.orchestra.run.smithi113.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-26T04:18:53.079 INFO:teuthology.orchestra.run.smithi113.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-26T04:18:53.203 INFO:teuthology.orchestra.run.smithi113.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-26T04:18:53.203 INFO:teuthology.orchestra.run.smithi113.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-26T04:18:53.203 INFO:teuthology.orchestra.run.smithi113.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-26T04:18:53.210 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-26T04:18:53.214 INFO:teuthology.orchestra.run.smithi113.stderr:done 2024-04-26T04:18:53.219 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-26T04:18:53.230 DEBUG:teuthology.orchestra.run.smithi113:> sudo shutdown -r now 2024-04-26T04:18:53.259 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-26T04:18:53.265 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-26T04:18:53.391 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-26T04:18:53.391 INFO:teuthology.orchestra.run.smithi012.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-26T04:18:53.392 INFO:teuthology.orchestra.run.smithi012.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-26T04:18:53.412 INFO:teuthology.orchestra.run.smithi012.stderr:done 2024-04-26T04:18:53.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2024-04-26T04:19:21.130 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-04-26T04:19:21.131 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-26T04:19:21.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:19:21.308 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-04-26T04:19:21.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-26T04:19:21.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:19:23.234 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-26T04:19:23.234 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-26T04:19:23.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:19:23.426 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-26T04:19:23.426 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-26T04:19:23.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:19:39.682 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-04-26T04:19:39.683 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-04-26T04:19:41.726 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2024-04-26T04:19:41.986 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-04-26T04:19:48.690 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-26T04:19:48.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:19:48.693 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-26T04:19:48.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:19:50.729 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-26T04:19:50.730 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:19:50.987 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-26T04:19:50.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:19:51.742 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-04-26T04:19:51.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-04-26T04:19:53.790 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2024-04-26T04:20:03.751 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-26T04:20:03.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:20:03.774 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-26T04:20:03.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:20:05.793 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-26T04:20:05.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:20:06.814 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-04-26T04:20:08.862 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2024-04-26T04:20:09.374 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-04-26T04:20:21.386 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-26T04:20:21.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:20:21.817 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-26T04:20:21.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:20:22.178 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.137 2024-04-26T04:20:23.865 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-26T04:20:23.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:20:24.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-04-26T04:20:37.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-26T04:20:37.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:20:37.558 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-04-26T04:20:38.724 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-26T04:20:38.725 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.15.0-105-generic"... 2024-04-26T04:20:38.725 DEBUG:teuthology.orchestra.run.smithi137:> uname -r 2024-04-26T04:20:38.774 INFO:teuthology.orchestra.run.smithi137.stdout:5.15.0-105-generic 2024-04-26T04:20:38.774 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-26T04:20:38.774 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T04:20:38.774 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-26T04:20:39.447 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-26T04:20:39.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:20:39.755 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-04-26T04:20:39.775 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-26T04:20:39.775 DEBUG:teuthology.orchestra.run.smithi137:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T04:20:39.929 INFO:teuthology.orchestra.run.smithi137.stdout:ttyS1 2024-04-26T04:20:39.955 DEBUG:teuthology.parallel:result is None 2024-04-26T04:20:40.350 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-04-26T04:20:40.422 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-26T04:20:40.422 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-26T04:20:40.423 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-04-26T04:20:40.474 INFO:teuthology.orchestra.run.smithi012.stdout:5.15.0-105-generic 2024-04-26T04:20:40.474 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-26T04:20:40.474 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T04:20:40.475 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-26T04:20:41.476 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-26T04:20:41.476 DEBUG:teuthology.orchestra.run.smithi012:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T04:20:41.550 INFO:teuthology.orchestra.run.smithi012.stdout:ttyS1 2024-04-26T04:20:41.576 DEBUG:teuthology.parallel:result is None 2024-04-26T04:20:42.402 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2024-04-26T04:20:58.351 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-26T04:20:58.352 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:20:58.635 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-04-26T04:20:59.181 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-26T04:20:59.181 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.15.0-105-generic"... 2024-04-26T04:20:59.181 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-04-26T04:20:59.227 INFO:teuthology.orchestra.run.smithi178.stdout:5.15.0-105-generic 2024-04-26T04:20:59.227 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-26T04:20:59.227 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T04:20:59.227 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-26T04:21:00.227 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-04-26T04:21:00.228 DEBUG:teuthology.orchestra.run.smithi178:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T04:21:00.403 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-26T04:21:00.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:21:00.576 INFO:teuthology.orchestra.run.smithi178.stdout:ttyS1 2024-04-26T04:21:00.616 DEBUG:teuthology.parallel:result is None 2024-04-26T04:21:00.779 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-04-26T04:21:01.336 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-26T04:21:01.336 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-26T04:21:01.336 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2024-04-26T04:21:01.383 INFO:teuthology.orchestra.run.smithi113.stdout:5.15.0-105-generic 2024-04-26T04:21:01.383 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-26T04:21:01.383 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T04:21:01.383 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-26T04:21:02.384 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-26T04:21:02.384 DEBUG:teuthology.orchestra.run.smithi113:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T04:21:02.750 INFO:teuthology.orchestra.run.smithi113.stdout:ttyS1 2024-04-26T04:21:02.772 DEBUG:teuthology.parallel:result is None 2024-04-26T04:21:02.772 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T04:21:02.782 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T04:21:02.782 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T04:21:02.784 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T04:21:02.786 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T04:21:02.789 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T04:21:02.795 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T04:21:02.806 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T04:21:02.820 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T04:21:02.820 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T04:21:02.832 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T04:21:02.836 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T04:21:02.838 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T04:21:02.914 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T04:21:02.923 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T04:21:02.923 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T04:21:02.926 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T04:21:02.928 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T04:21:02.931 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T04:21:02.945 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:21:02.946 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:21:02.950 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:21:02.954 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:21:02.956 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:21:02.959 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:21:02.976 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:21:02.985 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:21:02.986 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T04:21:02.994 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T04:21:02.994 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T04:21:03.003 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T04:21:03.005 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T04:21:03.008 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T04:21:03.033 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T04:21:03.049 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T04:21:03.049 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T04:21:03.060 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T04:21:03.072 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T04:21:03.077 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T04:21:03.082 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T04:21:03.111 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T04:21:03.159 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:21:03.159 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T04:21:03.225 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T04:21:03.231 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T04:21:03.279 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:21:03.279 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T04:21:03.350 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T04:21:03.356 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T04:21:03.405 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:21:03.405 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T04:21:03.488 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T04:21:03.494 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T04:21:03.543 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:21:03.543 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T04:21:03.608 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-04-26T04:21:03.610 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2024-04-26T04:21:03.612 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-04-26T04:21:03.614 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-04-26T04:21:03.680 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T04:21:03.687 INFO:teuthology.task.internal:Starting timer... 2024-04-26T04:21:03.687 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T04:21:03.700 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T04:21:03.713 DEBUG:teuthology.task.selinux:Excluding smithi012: OS 'ubuntu' does not support SELinux 2024-04-26T04:21:03.713 DEBUG:teuthology.task.selinux:Excluding smithi113: OS 'ubuntu' does not support SELinux 2024-04-26T04:21:03.714 DEBUG:teuthology.task.selinux:Excluding smithi137: OS 'ubuntu' does not support SELinux 2024-04-26T04:21:03.714 DEBUG:teuthology.task.selinux:Excluding smithi178: OS 'ubuntu' does not support SELinux 2024-04-26T04:21:03.714 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T04:21:03.714 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-26T04:21:03.714 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T04:21:03.714 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T04:21:03.727 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T04:21:03.742 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T04:21:03.744 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi113.front.sepia.ceph.com,smithi137.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T04:28:32.104 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi113.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2024-04-26T04:28:32.105 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-26T04:28:32.106 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:28:32.195 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-04-26T04:28:32.282 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-04-26T04:28:32.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-26T04:28:32.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:28:32.366 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-04-26T04:28:32.438 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-04-26T04:28:32.438 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-26T04:28:32.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:28:32.526 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-04-26T04:28:32.602 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-26T04:28:32.602 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-26T04:28:32.603 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:28:32.685 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-04-26T04:28:32.766 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-04-26T04:28:32.767 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T04:28:32.777 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T04:28:32.777 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T04:28:32.778 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:28:32.781 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T04:28:32.781 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:28:32.783 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T04:28:32.783 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:28:32.785 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T04:28:32.785 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:28:32.811 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-26T04:28:32.811 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: Command line: ntpd -gq 2024-04-26T04:28:32.812 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: ---------------------------------------------------- 2024-04-26T04:28:32.812 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: ntp-4 is maintained by Network Time Foundation, 2024-04-26T04:28:32.812 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-26T04:28:32.812 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: corporation. Support and training for ntp-4 are 2024-04-26T04:28:32.812 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: available at https://www.nwtime.org/support 2024-04-26T04:28:32.812 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: ---------------------------------------------------- 2024-04-26T04:28:32.812 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: proto: precision = 0.046 usec (-24) 2024-04-26T04:28:32.812 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: basedate set to 2022-02-04 2024-04-26T04:28:32.812 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: gps base set to 2022-02-06 (week 2196) 2024-04-26T04:28:32.813 INFO:teuthology.orchestra.run.smithi012.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-26T04:28:32.813 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-26T04:28:32.813 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: restrict ::: KOD does nothing without LIMITED. 2024-04-26T04:28:32.813 INFO:teuthology.orchestra.run.smithi012.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-26T04:28:32.814 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: Listen and drop on 0 v6wildcard [::]:123 2024-04-26T04:28:32.814 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-26T04:28:32.815 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: Listen normally on 2 lo 127.0.0.1:123 2024-04-26T04:28:32.815 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: Listen normally on 3 ens1f0 172.21.15.12:123 2024-04-26T04:28:32.815 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: Listen normally on 4 lo [::1]:123 2024-04-26T04:28:32.815 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:25ea%4]:123 2024-04-26T04:28:32.815 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:32 ntpd[17137]: Listening on routing socket on fd #22 for interface updates 2024-04-26T04:28:32.817 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-26T04:28:32.817 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: Command line: ntpd -gq 2024-04-26T04:28:32.817 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: ---------------------------------------------------- 2024-04-26T04:28:32.817 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: ntp-4 is maintained by Network Time Foundation, 2024-04-26T04:28:32.817 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-26T04:28:32.817 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: corporation. Support and training for ntp-4 are 2024-04-26T04:28:32.817 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: available at https://www.nwtime.org/support 2024-04-26T04:28:32.817 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: ---------------------------------------------------- 2024-04-26T04:28:32.817 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: proto: precision = 0.063 usec (-24) 2024-04-26T04:28:32.817 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: basedate set to 2022-02-04 2024-04-26T04:28:32.818 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: gps base set to 2022-02-06 (week 2196) 2024-04-26T04:28:32.818 INFO:teuthology.orchestra.run.smithi137.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-26T04:28:32.818 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-26T04:28:32.818 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: Command line: ntpd -gq 2024-04-26T04:28:32.818 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: ---------------------------------------------------- 2024-04-26T04:28:32.818 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: ntp-4 is maintained by Network Time Foundation, 2024-04-26T04:28:32.818 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-26T04:28:32.818 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: corporation. Support and training for ntp-4 are 2024-04-26T04:28:32.819 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: available at https://www.nwtime.org/support 2024-04-26T04:28:32.819 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: ---------------------------------------------------- 2024-04-26T04:28:32.819 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: proto: precision = 0.045 usec (-24) 2024-04-26T04:28:32.819 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: basedate set to 2022-02-04 2024-04-26T04:28:32.819 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: gps base set to 2022-02-06 (week 2196) 2024-04-26T04:28:32.819 INFO:teuthology.orchestra.run.smithi113.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-26T04:28:32.819 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-26T04:28:32.820 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: restrict ::: KOD does nothing without LIMITED. 2024-04-26T04:28:32.820 INFO:teuthology.orchestra.run.smithi137.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-26T04:28:32.820 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-26T04:28:32.820 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: restrict ::: KOD does nothing without LIMITED. 2024-04-26T04:28:32.820 INFO:teuthology.orchestra.run.smithi113.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-26T04:28:32.823 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: Listen and drop on 0 v6wildcard [::]:123 2024-04-26T04:28:32.823 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-26T04:28:32.823 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: Listen normally on 2 lo 127.0.0.1:123 2024-04-26T04:28:32.823 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: Listen normally on 3 enp3s0f1 172.21.15.137:123 2024-04-26T04:28:32.823 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: Listen normally on 4 lo [::1]:123 2024-04-26T04:28:32.823 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8137%5]:123 2024-04-26T04:28:32.823 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:32 ntpd[17234]: Listening on routing socket on fd #22 for interface updates 2024-04-26T04:28:32.824 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: Listen and drop on 0 v6wildcard [::]:123 2024-04-26T04:28:32.824 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-26T04:28:32.824 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: Listen normally on 2 lo 127.0.0.1:123 2024-04-26T04:28:32.824 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: Listen normally on 3 enp3s0f1 172.21.15.113:123 2024-04-26T04:28:32.824 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: Listen normally on 4 lo [::1]:123 2024-04-26T04:28:32.824 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d7c1%5]:123 2024-04-26T04:28:32.824 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:32 ntpd[17194]: Listening on routing socket on fd #22 for interface updates 2024-04-26T04:28:32.841 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-26T04:28:32.841 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: Command line: ntpd -gq 2024-04-26T04:28:32.841 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: ---------------------------------------------------- 2024-04-26T04:28:32.841 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: ntp-4 is maintained by Network Time Foundation, 2024-04-26T04:28:32.841 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-26T04:28:32.841 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: corporation. Support and training for ntp-4 are 2024-04-26T04:28:32.841 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: available at https://www.nwtime.org/support 2024-04-26T04:28:32.841 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: ---------------------------------------------------- 2024-04-26T04:28:32.841 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: proto: precision = 0.046 usec (-24) 2024-04-26T04:28:32.842 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: basedate set to 2022-02-04 2024-04-26T04:28:32.842 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: gps base set to 2022-02-06 (week 2196) 2024-04-26T04:28:32.842 INFO:teuthology.orchestra.run.smithi178.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-26T04:28:32.843 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-26T04:28:32.843 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: restrict ::: KOD does nothing without LIMITED. 2024-04-26T04:28:32.843 INFO:teuthology.orchestra.run.smithi178.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-26T04:28:32.844 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: Listen and drop on 0 v6wildcard [::]:123 2024-04-26T04:28:32.844 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-26T04:28:32.844 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: Listen normally on 2 lo 127.0.0.1:123 2024-04-26T04:28:32.844 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: Listen normally on 3 enp3s0f1 172.21.15.178:123 2024-04-26T04:28:32.844 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: Listen normally on 4 lo [::1]:123 2024-04-26T04:28:32.844 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7ef9%5]:123 2024-04-26T04:28:32.844 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:32 ntpd[17208]: Listening on routing socket on fd #22 for interface updates 2024-04-26T04:28:33.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:33 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:33.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:33 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:33.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:33 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:33.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:33 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:33.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:33 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:33.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:33 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:33 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:33 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:33 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:33 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:33 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:33.813 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:33 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:33.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:33 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:33.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:33 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:33.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:33 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:33.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:33 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:33.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:33 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:33.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:33 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:35.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:35 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:35.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:35 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:35.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:35 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:35.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:35 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:35.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:35 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:35.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:35 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:35.814 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:35 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:35.815 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:35 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:35.815 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:35 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:35.815 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:35 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:35.815 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:35 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:35.815 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:35 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:35.816 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:35 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:35.816 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:35 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:35.816 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:35 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:35.816 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:35 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:35.816 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:35 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:35.816 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:35 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:35.817 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:35 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:35.817 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:35 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:35.817 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:35 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:35.817 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:35 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:35.817 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:35 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:35.817 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:35 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:35.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:35.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:35.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:35.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:35.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:35.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:35.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:35.843 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:35.843 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:35.843 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:35.843 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:35.843 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:35.843 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:35.843 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:35 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:36.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:36 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:36 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:36 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:36 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:36 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:36 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:36 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:36.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:36 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:36.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:36 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:36 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:36 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:36.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:36 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:36.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:36 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:36 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:36 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:36 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:36 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:36 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:36 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:36.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:36 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:36.815 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:36.816 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:36 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:36.841 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:36 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.841 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:36 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.841 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:36 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:36.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:36 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:36.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:36 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:36.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:36 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:37.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:37 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:37.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:37 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:37.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:37 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:37.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:37 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:37.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:37 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:37.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:37 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:37 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:37.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:37 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:37.813 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:37 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:37.813 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:37 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:37.813 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:37 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:37.813 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:37 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:37.813 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:37 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:37.813 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:37 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:37 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:37 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:37 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:37 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:37 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:37 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:37 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:37 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:37 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:37 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:37 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:37.815 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:37 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:37.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:37 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:37.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:37 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:37.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:37 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:37.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:37 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:37.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:37 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:37.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:37 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:37.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:37 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:37.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:37 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:37.843 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:37 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:37.843 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:37 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:37.843 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:37 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:37.843 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:37 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:38.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:38 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:38.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:38 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:38.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:38 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:38.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:38 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:38.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:38 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:38.813 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:38 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:38.813 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:38 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:38.813 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:38 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:38.813 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:38 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:38.813 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:38 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:38.813 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:38 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:38.813 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:38 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:38.815 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:38 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:38.816 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:38 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:38.816 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:38 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:38.816 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:38 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:38.816 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:38 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:38.816 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:38 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:38 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:38 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:38.817 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:38 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:38.818 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:38 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:38.818 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:38 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:38.818 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:38 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:38.818 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:38 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:38.818 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:38 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:38.818 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:38 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:38.818 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:38 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:38.819 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:38 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:38.819 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:38 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:38 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:38.841 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:38 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:38 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:38 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:38 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:38.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:38 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:39 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:39 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:39 ntpd[17137]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:39 ntpd[17137]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:39 ntpd[17137]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:39.811 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:39 ntpd[17137]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:39.812 INFO:teuthology.orchestra.run.smithi012.stdout:26 Apr 04:28:39 ntpd[17137]: ntpd: time slew -0.001410 s 2024-04-26T04:28:39.812 INFO:teuthology.orchestra.run.smithi012.stdout:ntpd: time slew -0.001410s 2024-04-26T04:28:39.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:39 ntpd[17137]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-26T04:28:39.812 INFO:teuthology.orchestra.run.smithi012.stderr:26 Apr 04:28:39 ntpd[17137]: can't open /var/log/ntpstats/loopstats.20240426: Permission denied 2024-04-26T04:28:39.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:39 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:39.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:39 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:39.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:39 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:39.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:39 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:39.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:39 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:39.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:39 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:39.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:39 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:39.814 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:39 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:39.815 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:39 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:39.815 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:39 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:39.815 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:39 ntpd[17194]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:39.815 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:39 ntpd[17194]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:39.815 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:39 ntpd[17194]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:39.815 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:39 ntpd[17194]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:39.816 INFO:teuthology.orchestra.run.smithi137.stdout:26 Apr 04:28:39 ntpd[17234]: ntpd: time slew +0.000050 s 2024-04-26T04:28:39.816 INFO:teuthology.orchestra.run.smithi137.stdout:ntpd: time slew +0.000050s 2024-04-26T04:28:39.816 INFO:teuthology.orchestra.run.smithi113.stdout:26 Apr 04:28:39 ntpd[17194]: ntpd: time slew +0.001047 s 2024-04-26T04:28:39.816 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd: time slew +0.001047s 2024-04-26T04:28:39.817 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:39 ntpd[17234]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-26T04:28:39.817 INFO:teuthology.orchestra.run.smithi137.stderr:26 Apr 04:28:39 ntpd[17234]: can't open /var/log/ntpstats/loopstats.20240426: Permission denied 2024-04-26T04:28:39.817 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:39 ntpd[17194]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-26T04:28:39.817 INFO:teuthology.orchestra.run.smithi113.stderr:26 Apr 04:28:39 ntpd[17194]: can't open /var/log/ntpstats/loopstats.20240426: Permission denied 2024-04-26T04:28:39.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:39 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:39.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:39 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:39.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:39 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:39.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:39 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:39.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:39 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:39.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:39 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:39.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:39 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:39.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:39 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:39.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:39 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:39.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:39 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:39.843 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:39 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:39.843 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:39 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:39.864 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2024-04-26T04:28:39.865 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2024-04-26T04:28:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:39.865 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:39.868 INFO:teuthology.orchestra.run.smithi113.stdout: remote refid st t when poll reach delay offset jitter 2024-04-26T04:28:39.868 INFO:teuthology.orchestra.run.smithi113.stdout:============================================================================== 2024-04-26T04:28:39.868 INFO:teuthology.orchestra.run.smithi113.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:39.868 INFO:teuthology.orchestra.run.smithi113.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:39.868 INFO:teuthology.orchestra.run.smithi113.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:39.868 INFO:teuthology.orchestra.run.smithi113.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:39.874 INFO:teuthology.orchestra.run.smithi137.stdout: remote refid st t when poll reach delay offset jitter 2024-04-26T04:28:39.874 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================== 2024-04-26T04:28:39.874 INFO:teuthology.orchestra.run.smithi137.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:39.874 INFO:teuthology.orchestra.run.smithi137.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:39.874 INFO:teuthology.orchestra.run.smithi137.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:39.874 INFO:teuthology.orchestra.run.smithi137.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:40.841 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:40 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:40.841 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:40 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:40.841 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:40 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:40.841 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:40 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:40.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:40 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:40.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:40 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:41.841 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:41 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:41.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:41 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:41.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:41 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:41.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:41 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:41.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:41 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-26T04:28:41.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:41 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240426: Permission denied 2024-04-26T04:28:41.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:41 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-26T04:28:41.842 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:41 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240426: Permission denied 2024-04-26T04:28:41.842 INFO:teuthology.orchestra.run.smithi178.stdout:26 Apr 04:28:41 ntpd[17208]: ntpd: time slew -0.001942 s 2024-04-26T04:28:41.842 INFO:teuthology.orchestra.run.smithi178.stdout:ntpd: time slew -0.001942s 2024-04-26T04:28:41.844 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:41 ntpd[17208]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-26T04:28:41.844 INFO:teuthology.orchestra.run.smithi178.stderr:26 Apr 04:28:41 ntpd[17208]: can't open /var/log/ntpstats/loopstats.20240426: Permission denied 2024-04-26T04:28:41.895 INFO:teuthology.orchestra.run.smithi178.stdout: remote refid st t when poll reach delay offset jitter 2024-04-26T04:28:41.895 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================== 2024-04-26T04:28:41.895 INFO:teuthology.orchestra.run.smithi178.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:41.895 INFO:teuthology.orchestra.run.smithi178.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:41.895 INFO:teuthology.orchestra.run.smithi178.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:41.895 INFO:teuthology.orchestra.run.smithi178.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:28:41.896 INFO:teuthology.run_tasks:Running task install... 2024-04-26T04:28:41.906 DEBUG:teuthology.task.install:project ceph 2024-04-26T04:28:41.906 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}} 2024-04-26T04:28:41.906 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'} 2024-04-26T04:28:41.906 INFO:teuthology.task.install:Using flavor: default 2024-04-26T04:28:41.913 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T04:28:41.913 INFO:teuthology.task.install:extra packages: [] 2024-04-26T04:28:41.917 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-key list | grep Ceph 2024-04-26T04:28:41.917 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-key list | grep Ceph 2024-04-26T04:28:41.917 DEBUG:teuthology.orchestra.run.smithi137:> sudo apt-key list | grep Ceph 2024-04-26T04:28:41.918 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-key list | grep Ceph 2024-04-26T04:28:41.979 INFO:teuthology.orchestra.run.smithi113.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-26T04:28:41.986 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-26T04:28:41.987 INFO:teuthology.orchestra.run.smithi137.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-26T04:28:41.994 INFO:teuthology.orchestra.run.smithi178.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-26T04:28:42.008 INFO:teuthology.orchestra.run.smithi113.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-26T04:28:42.008 INFO:teuthology.orchestra.run.smithi113.stdout:uid [ unknown] Ceph.com (release key) 2024-04-26T04:28:42.008 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-26T04:28:42.009 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T04:28:42.017 INFO:teuthology.orchestra.run.smithi137.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-26T04:28:42.017 INFO:teuthology.orchestra.run.smithi137.stdout:uid [ unknown] Ceph.com (release key) 2024-04-26T04:28:42.019 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-26T04:28:42.019 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph.com (release key) 2024-04-26T04:28:42.019 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-26T04:28:42.019 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T04:28:42.021 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-26T04:28:42.021 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T04:28:42.028 INFO:teuthology.orchestra.run.smithi178.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-26T04:28:42.028 INFO:teuthology.orchestra.run.smithi178.stdout:uid [ unknown] Ceph.com (release key) 2024-04-26T04:28:42.028 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-26T04:28:42.028 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T04:28:42.181 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default/ 2024-04-26T04:28:42.181 INFO:teuthology.task.install.deb:Package version is 19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:28:42.186 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default/ 2024-04-26T04:28:42.186 INFO:teuthology.task.install.deb:Package version is 19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:28:42.193 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default/ 2024-04-26T04:28:42.193 INFO:teuthology.task.install.deb:Package version is 19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:28:42.253 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default/ 2024-04-26T04:28:42.253 INFO:teuthology.task.install.deb:Package version is 19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:28:42.360 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:28:42.360 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-26T04:28:42.363 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:28:42.363 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-26T04:28:42.366 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:28:42.366 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-26T04:28:42.370 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:28:42.370 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-26T04:28:42.380 DEBUG:teuthology.orchestra.run.smithi178:> sudo apt-get update 2024-04-26T04:28:42.384 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-get update 2024-04-26T04:28:42.385 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-04-26T04:28:42.409 DEBUG:teuthology.orchestra.run.smithi137:> sudo apt-get update 2024-04-26T04:28:42.555 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-26T04:28:42.559 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-26T04:28:42.560 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-26T04:28:42.560 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-26T04:28:42.560 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-26T04:28:42.569 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-26T04:28:42.573 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-26T04:28:42.575 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-26T04:28:42.584 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-26T04:28:42.589 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-26T04:28:42.593 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-26T04:28:42.595 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-26T04:28:42.605 INFO:teuthology.orchestra.run.smithi113.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy InRelease 2024-04-26T04:28:42.626 INFO:teuthology.orchestra.run.smithi113.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-26T04:28:42.631 INFO:teuthology.orchestra.run.smithi137.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy InRelease 2024-04-26T04:28:42.648 INFO:teuthology.orchestra.run.smithi113.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-26T04:28:42.652 INFO:teuthology.orchestra.run.smithi137.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-26T04:28:42.668 INFO:teuthology.orchestra.run.smithi178.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy InRelease 2024-04-26T04:28:42.669 INFO:teuthology.orchestra.run.smithi113.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,421 B] 2024-04-26T04:28:42.670 INFO:teuthology.orchestra.run.smithi178.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-26T04:28:42.672 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy InRelease 2024-04-26T04:28:42.674 INFO:teuthology.orchestra.run.smithi137.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-26T04:28:42.696 INFO:teuthology.orchestra.run.smithi137.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-26T04:28:42.696 INFO:teuthology.orchestra.run.smithi113.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-26T04:28:42.704 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-26T04:28:42.705 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-26T04:28:42.709 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-26T04:28:42.725 INFO:teuthology.orchestra.run.smithi137.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,421 B] 2024-04-26T04:28:42.742 INFO:teuthology.orchestra.run.smithi178.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-26T04:28:42.745 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-26T04:28:42.778 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-26T04:28:42.782 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,421 B] 2024-04-26T04:28:42.783 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-26T04:28:42.820 INFO:teuthology.orchestra.run.smithi178.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,421 B] 2024-04-26T04:28:42.824 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-26T04:28:42.862 INFO:teuthology.orchestra.run.smithi137.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-26T04:28:42.907 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 28.9 kB in 0s (84.3 kB/s) 2024-04-26T04:28:42.908 INFO:teuthology.orchestra.run.smithi113.stdout:Fetched 28.9 kB in 0s (83.9 kB/s) 2024-04-26T04:28:42.935 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 28.9 kB in 0s (78.7 kB/s) 2024-04-26T04:28:43.005 INFO:teuthology.orchestra.run.smithi137.stdout:Fetched 28.9 kB in 0s (70.0 kB/s) 2024-04-26T04:28:43.867 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-04-26T04:28:43.882 DEBUG:teuthology.orchestra.run.smithi113:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2457-gddeaa917-1jammy cephadm=19.0.0-2457-gddeaa917-1jammy ceph-mds=19.0.0-2457-gddeaa917-1jammy ceph-mgr=19.0.0-2457-gddeaa917-1jammy ceph-common=19.0.0-2457-gddeaa917-1jammy ceph-fuse=19.0.0-2457-gddeaa917-1jammy ceph-test=19.0.0-2457-gddeaa917-1jammy ceph-volume=19.0.0-2457-gddeaa917-1jammy radosgw=19.0.0-2457-gddeaa917-1jammy python3-rados=19.0.0-2457-gddeaa917-1jammy python3-rgw=19.0.0-2457-gddeaa917-1jammy python3-cephfs=19.0.0-2457-gddeaa917-1jammy python3-rbd=19.0.0-2457-gddeaa917-1jammy libcephfs2=19.0.0-2457-gddeaa917-1jammy libcephfs-dev=19.0.0-2457-gddeaa917-1jammy librados2=19.0.0-2457-gddeaa917-1jammy librbd1=19.0.0-2457-gddeaa917-1jammy rbd-fuse=19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:28:43.908 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-04-26T04:28:43.912 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-04-26T04:28:43.924 DEBUG:teuthology.orchestra.run.smithi178:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2457-gddeaa917-1jammy cephadm=19.0.0-2457-gddeaa917-1jammy ceph-mds=19.0.0-2457-gddeaa917-1jammy ceph-mgr=19.0.0-2457-gddeaa917-1jammy ceph-common=19.0.0-2457-gddeaa917-1jammy ceph-fuse=19.0.0-2457-gddeaa917-1jammy ceph-test=19.0.0-2457-gddeaa917-1jammy ceph-volume=19.0.0-2457-gddeaa917-1jammy radosgw=19.0.0-2457-gddeaa917-1jammy python3-rados=19.0.0-2457-gddeaa917-1jammy python3-rgw=19.0.0-2457-gddeaa917-1jammy python3-cephfs=19.0.0-2457-gddeaa917-1jammy python3-rbd=19.0.0-2457-gddeaa917-1jammy libcephfs2=19.0.0-2457-gddeaa917-1jammy libcephfs-dev=19.0.0-2457-gddeaa917-1jammy librados2=19.0.0-2457-gddeaa917-1jammy librbd1=19.0.0-2457-gddeaa917-1jammy rbd-fuse=19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:28:43.927 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2457-gddeaa917-1jammy cephadm=19.0.0-2457-gddeaa917-1jammy ceph-mds=19.0.0-2457-gddeaa917-1jammy ceph-mgr=19.0.0-2457-gddeaa917-1jammy ceph-common=19.0.0-2457-gddeaa917-1jammy ceph-fuse=19.0.0-2457-gddeaa917-1jammy ceph-test=19.0.0-2457-gddeaa917-1jammy ceph-volume=19.0.0-2457-gddeaa917-1jammy radosgw=19.0.0-2457-gddeaa917-1jammy python3-rados=19.0.0-2457-gddeaa917-1jammy python3-rgw=19.0.0-2457-gddeaa917-1jammy python3-cephfs=19.0.0-2457-gddeaa917-1jammy python3-rbd=19.0.0-2457-gddeaa917-1jammy libcephfs2=19.0.0-2457-gddeaa917-1jammy libcephfs-dev=19.0.0-2457-gddeaa917-1jammy librados2=19.0.0-2457-gddeaa917-1jammy librbd1=19.0.0-2457-gddeaa917-1jammy rbd-fuse=19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:28:43.943 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-04-26T04:28:43.984 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-04-26T04:28:43.984 INFO:teuthology.orchestra.run.smithi178.stdout:Reading package lists... 2024-04-26T04:28:43.989 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-04-26T04:28:44.011 DEBUG:teuthology.orchestra.run.smithi137:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2457-gddeaa917-1jammy cephadm=19.0.0-2457-gddeaa917-1jammy ceph-mds=19.0.0-2457-gddeaa917-1jammy ceph-mgr=19.0.0-2457-gddeaa917-1jammy ceph-common=19.0.0-2457-gddeaa917-1jammy ceph-fuse=19.0.0-2457-gddeaa917-1jammy ceph-test=19.0.0-2457-gddeaa917-1jammy ceph-volume=19.0.0-2457-gddeaa917-1jammy radosgw=19.0.0-2457-gddeaa917-1jammy python3-rados=19.0.0-2457-gddeaa917-1jammy python3-rgw=19.0.0-2457-gddeaa917-1jammy python3-cephfs=19.0.0-2457-gddeaa917-1jammy python3-rbd=19.0.0-2457-gddeaa917-1jammy libcephfs2=19.0.0-2457-gddeaa917-1jammy libcephfs-dev=19.0.0-2457-gddeaa917-1jammy librados2=19.0.0-2457-gddeaa917-1jammy librbd1=19.0.0-2457-gddeaa917-1jammy rbd-fuse=19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:28:44.063 INFO:teuthology.orchestra.run.smithi137.stdout:Reading package lists... 2024-04-26T04:28:44.142 INFO:teuthology.orchestra.run.smithi113.stdout:Building dependency tree... 2024-04-26T04:28:44.143 INFO:teuthology.orchestra.run.smithi113.stdout:Reading state information... 2024-04-26T04:28:44.179 INFO:teuthology.orchestra.run.smithi178.stdout:Building dependency tree... 2024-04-26T04:28:44.179 INFO:teuthology.orchestra.run.smithi178.stdout:Reading state information... 2024-04-26T04:28:44.189 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-04-26T04:28:44.189 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-04-26T04:28:44.242 INFO:teuthology.orchestra.run.smithi137.stdout:Building dependency tree... 2024-04-26T04:28:44.242 INFO:teuthology.orchestra.run.smithi137.stdout:Reading state information... 2024-04-26T04:28:44.325 INFO:teuthology.orchestra.run.smithi113.stdout:The following packages were automatically installed and are no longer required: 2024-04-26T04:28:44.325 INFO:teuthology.orchestra.run.smithi113.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-26T04:28:44.325 INFO:teuthology.orchestra.run.smithi113.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-26T04:28:44.325 INFO:teuthology.orchestra.run.smithi113.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-26T04:28:44.326 INFO:teuthology.orchestra.run.smithi113.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-26T04:28:44.326 INFO:teuthology.orchestra.run.smithi113.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-26T04:28:44.326 INFO:teuthology.orchestra.run.smithi113.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-26T04:28:44.327 INFO:teuthology.orchestra.run.smithi113.stdout:The following additional packages will be installed: 2024-04-26T04:28:44.327 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-26T04:28:44.327 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-26T04:28:44.328 INFO:teuthology.orchestra.run.smithi113.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-26T04:28:44.328 INFO:teuthology.orchestra.run.smithi113.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-26T04:28:44.328 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-26T04:28:44.328 INFO:teuthology.orchestra.run.smithi113.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-26T04:28:44.328 INFO:teuthology.orchestra.run.smithi113.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-26T04:28:44.328 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-26T04:28:44.328 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-26T04:28:44.328 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-26T04:28:44.328 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-26T04:28:44.328 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-26T04:28:44.328 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-26T04:28:44.328 INFO:teuthology.orchestra.run.smithi113.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-26T04:28:44.329 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-26T04:28:44.329 INFO:teuthology.orchestra.run.smithi113.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-26T04:28:44.329 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-26T04:28:44.330 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-26T04:28:44.330 INFO:teuthology.orchestra.run.smithi113.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-26T04:28:44.330 INFO:teuthology.orchestra.run.smithi113.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-26T04:28:44.330 INFO:teuthology.orchestra.run.smithi113.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-26T04:28:44.330 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-26T04:28:44.330 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-26T04:28:44.330 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet zip 2024-04-26T04:28:44.331 INFO:teuthology.orchestra.run.smithi113.stdout:Suggested packages: 2024-04-26T04:28:44.331 INFO:teuthology.orchestra.run.smithi113.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-26T04:28:44.331 INFO:teuthology.orchestra.run.smithi113.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-26T04:28:44.331 INFO:teuthology.orchestra.run.smithi113.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-26T04:28:44.331 INFO:teuthology.orchestra.run.smithi113.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-26T04:28:44.331 INFO:teuthology.orchestra.run.smithi113.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-26T04:28:44.331 INFO:teuthology.orchestra.run.smithi113.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-26T04:28:44.331 INFO:teuthology.orchestra.run.smithi113.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-26T04:28:44.331 INFO:teuthology.orchestra.run.smithi113.stdout:Recommended packages: 2024-04-26T04:28:44.331 INFO:teuthology.orchestra.run.smithi113.stdout: btrfs-tools 2024-04-26T04:28:44.354 INFO:teuthology.orchestra.run.smithi178.stdout:The following packages were automatically installed and are no longer required: 2024-04-26T04:28:44.354 INFO:teuthology.orchestra.run.smithi178.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-26T04:28:44.355 INFO:teuthology.orchestra.run.smithi178.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-26T04:28:44.355 INFO:teuthology.orchestra.run.smithi178.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-26T04:28:44.355 INFO:teuthology.orchestra.run.smithi178.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-26T04:28:44.357 INFO:teuthology.orchestra.run.smithi178.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-26T04:28:44.357 INFO:teuthology.orchestra.run.smithi178.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-26T04:28:44.357 INFO:teuthology.orchestra.run.smithi178.stdout:The following additional packages will be installed: 2024-04-26T04:28:44.357 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-26T04:28:44.357 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-26T04:28:44.357 INFO:teuthology.orchestra.run.smithi178.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-26T04:28:44.357 INFO:teuthology.orchestra.run.smithi178.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-26T04:28:44.357 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-26T04:28:44.357 INFO:teuthology.orchestra.run.smithi178.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-26T04:28:44.357 INFO:teuthology.orchestra.run.smithi178.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-26T04:28:44.357 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-26T04:28:44.357 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-26T04:28:44.358 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-26T04:28:44.359 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet zip 2024-04-26T04:28:44.359 INFO:teuthology.orchestra.run.smithi178.stdout:Suggested packages: 2024-04-26T04:28:44.359 INFO:teuthology.orchestra.run.smithi178.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-26T04:28:44.359 INFO:teuthology.orchestra.run.smithi178.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-26T04:28:44.359 INFO:teuthology.orchestra.run.smithi178.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-26T04:28:44.359 INFO:teuthology.orchestra.run.smithi178.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-26T04:28:44.359 INFO:teuthology.orchestra.run.smithi178.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-26T04:28:44.359 INFO:teuthology.orchestra.run.smithi178.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-26T04:28:44.359 INFO:teuthology.orchestra.run.smithi178.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-26T04:28:44.360 INFO:teuthology.orchestra.run.smithi178.stdout:Recommended packages: 2024-04-26T04:28:44.360 INFO:teuthology.orchestra.run.smithi178.stdout: btrfs-tools 2024-04-26T04:28:44.371 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2024-04-26T04:28:44.371 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-26T04:28:44.371 INFO:teuthology.orchestra.run.smithi012.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-26T04:28:44.372 INFO:teuthology.orchestra.run.smithi012.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-26T04:28:44.372 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-26T04:28:44.373 INFO:teuthology.orchestra.run.smithi012.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-26T04:28:44.373 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-26T04:28:44.373 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2024-04-26T04:28:44.373 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-26T04:28:44.373 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-26T04:28:44.373 INFO:teuthology.orchestra.run.smithi012.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-26T04:28:44.373 INFO:teuthology.orchestra.run.smithi012.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-26T04:28:44.374 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-26T04:28:44.374 INFO:teuthology.orchestra.run.smithi012.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-26T04:28:44.375 INFO:teuthology.orchestra.run.smithi012.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-26T04:28:44.376 INFO:teuthology.orchestra.run.smithi012.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-26T04:28:44.376 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-26T04:28:44.376 INFO:teuthology.orchestra.run.smithi012.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-26T04:28:44.376 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet zip 2024-04-26T04:28:44.376 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2024-04-26T04:28:44.376 INFO:teuthology.orchestra.run.smithi012.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-26T04:28:44.376 INFO:teuthology.orchestra.run.smithi012.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-26T04:28:44.376 INFO:teuthology.orchestra.run.smithi012.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-26T04:28:44.377 INFO:teuthology.orchestra.run.smithi012.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-26T04:28:44.377 INFO:teuthology.orchestra.run.smithi012.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-26T04:28:44.377 INFO:teuthology.orchestra.run.smithi012.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-26T04:28:44.377 INFO:teuthology.orchestra.run.smithi012.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-26T04:28:44.377 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2024-04-26T04:28:44.377 INFO:teuthology.orchestra.run.smithi012.stdout: btrfs-tools 2024-04-26T04:28:44.420 INFO:teuthology.orchestra.run.smithi137.stdout:The following packages were automatically installed and are no longer required: 2024-04-26T04:28:44.420 INFO:teuthology.orchestra.run.smithi137.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-26T04:28:44.420 INFO:teuthology.orchestra.run.smithi137.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-26T04:28:44.420 INFO:teuthology.orchestra.run.smithi137.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-26T04:28:44.420 INFO:teuthology.orchestra.run.smithi137.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-26T04:28:44.420 INFO:teuthology.orchestra.run.smithi137.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-26T04:28:44.420 INFO:teuthology.orchestra.run.smithi137.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-26T04:28:44.420 INFO:teuthology.orchestra.run.smithi137.stdout:The following additional packages will be installed: 2024-04-26T04:28:44.420 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-26T04:28:44.420 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-26T04:28:44.421 INFO:teuthology.orchestra.run.smithi137.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-26T04:28:44.422 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-26T04:28:44.422 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-26T04:28:44.422 INFO:teuthology.orchestra.run.smithi137.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-26T04:28:44.422 INFO:teuthology.orchestra.run.smithi137.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-26T04:28:44.422 INFO:teuthology.orchestra.run.smithi137.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-26T04:28:44.423 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-26T04:28:44.423 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-26T04:28:44.423 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet zip 2024-04-26T04:28:44.423 INFO:teuthology.orchestra.run.smithi137.stdout:Suggested packages: 2024-04-26T04:28:44.423 INFO:teuthology.orchestra.run.smithi137.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-26T04:28:44.423 INFO:teuthology.orchestra.run.smithi137.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-26T04:28:44.423 INFO:teuthology.orchestra.run.smithi137.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-26T04:28:44.423 INFO:teuthology.orchestra.run.smithi137.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-26T04:28:44.423 INFO:teuthology.orchestra.run.smithi137.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-26T04:28:44.424 INFO:teuthology.orchestra.run.smithi137.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-26T04:28:44.424 INFO:teuthology.orchestra.run.smithi137.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-26T04:28:44.424 INFO:teuthology.orchestra.run.smithi137.stdout:Recommended packages: 2024-04-26T04:28:44.424 INFO:teuthology.orchestra.run.smithi137.stdout: btrfs-tools 2024-04-26T04:28:44.428 INFO:teuthology.orchestra.run.smithi113.stdout:The following NEW packages will be installed: 2024-04-26T04:28:44.428 INFO:teuthology.orchestra.run.smithi113.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-26T04:28:44.428 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-26T04:28:44.430 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-26T04:28:44.430 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-26T04:28:44.430 INFO:teuthology.orchestra.run.smithi113.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-26T04:28:44.430 INFO:teuthology.orchestra.run.smithi113.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-26T04:28:44.430 INFO:teuthology.orchestra.run.smithi113.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-26T04:28:44.430 INFO:teuthology.orchestra.run.smithi113.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-26T04:28:44.430 INFO:teuthology.orchestra.run.smithi113.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-26T04:28:44.430 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-26T04:28:44.430 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-26T04:28:44.431 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-26T04:28:44.431 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-26T04:28:44.431 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-26T04:28:44.431 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-26T04:28:44.431 INFO:teuthology.orchestra.run.smithi113.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-26T04:28:44.431 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-26T04:28:44.431 INFO:teuthology.orchestra.run.smithi113.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-26T04:28:44.431 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-26T04:28:44.431 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-26T04:28:44.431 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-26T04:28:44.432 INFO:teuthology.orchestra.run.smithi113.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-26T04:28:44.432 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-26T04:28:44.432 INFO:teuthology.orchestra.run.smithi113.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-26T04:28:44.432 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-26T04:28:44.432 INFO:teuthology.orchestra.run.smithi113.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-26T04:28:44.458 INFO:teuthology.orchestra.run.smithi178.stdout:The following NEW packages will be installed: 2024-04-26T04:28:44.458 INFO:teuthology.orchestra.run.smithi178.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-26T04:28:44.458 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-26T04:28:44.459 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-26T04:28:44.459 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-26T04:28:44.459 INFO:teuthology.orchestra.run.smithi178.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-26T04:28:44.459 INFO:teuthology.orchestra.run.smithi178.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-26T04:28:44.459 INFO:teuthology.orchestra.run.smithi178.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-26T04:28:44.460 INFO:teuthology.orchestra.run.smithi178.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-26T04:28:44.460 INFO:teuthology.orchestra.run.smithi178.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-26T04:28:44.460 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-26T04:28:44.460 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-26T04:28:44.460 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-26T04:28:44.460 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-26T04:28:44.460 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-26T04:28:44.461 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-26T04:28:44.461 INFO:teuthology.orchestra.run.smithi178.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-26T04:28:44.461 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-26T04:28:44.461 INFO:teuthology.orchestra.run.smithi178.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-26T04:28:44.461 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-26T04:28:44.461 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-26T04:28:44.461 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-26T04:28:44.461 INFO:teuthology.orchestra.run.smithi178.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-26T04:28:44.461 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-26T04:28:44.461 INFO:teuthology.orchestra.run.smithi178.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-26T04:28:44.461 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-26T04:28:44.461 INFO:teuthology.orchestra.run.smithi178.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-26T04:28:44.477 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2024-04-26T04:28:44.477 INFO:teuthology.orchestra.run.smithi012.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-26T04:28:44.477 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-26T04:28:44.477 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-26T04:28:44.477 INFO:teuthology.orchestra.run.smithi012.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-26T04:28:44.477 INFO:teuthology.orchestra.run.smithi012.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-26T04:28:44.477 INFO:teuthology.orchestra.run.smithi012.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-26T04:28:44.478 INFO:teuthology.orchestra.run.smithi012.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-26T04:28:44.479 INFO:teuthology.orchestra.run.smithi012.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-26T04:28:44.479 INFO:teuthology.orchestra.run.smithi012.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-26T04:28:44.479 INFO:teuthology.orchestra.run.smithi012.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-26T04:28:44.479 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-26T04:28:44.479 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-26T04:28:44.479 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-26T04:28:44.479 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-26T04:28:44.479 INFO:teuthology.orchestra.run.smithi012.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-26T04:28:44.479 INFO:teuthology.orchestra.run.smithi012.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-26T04:28:44.479 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-26T04:28:44.479 INFO:teuthology.orchestra.run.smithi012.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-26T04:28:44.479 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-26T04:28:44.479 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-26T04:28:44.480 INFO:teuthology.orchestra.run.smithi012.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-26T04:28:44.480 INFO:teuthology.orchestra.run.smithi012.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-26T04:28:44.480 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-26T04:28:44.480 INFO:teuthology.orchestra.run.smithi012.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-26T04:28:44.480 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-26T04:28:44.480 INFO:teuthology.orchestra.run.smithi012.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-26T04:28:44.519 INFO:teuthology.orchestra.run.smithi137.stdout:The following NEW packages will be installed: 2024-04-26T04:28:44.519 INFO:teuthology.orchestra.run.smithi137.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-26T04:28:44.519 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-26T04:28:44.520 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-26T04:28:44.520 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-26T04:28:44.520 INFO:teuthology.orchestra.run.smithi137.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-26T04:28:44.520 INFO:teuthology.orchestra.run.smithi137.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-26T04:28:44.521 INFO:teuthology.orchestra.run.smithi137.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-26T04:28:44.521 INFO:teuthology.orchestra.run.smithi137.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-26T04:28:44.521 INFO:teuthology.orchestra.run.smithi137.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-26T04:28:44.521 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-26T04:28:44.522 INFO:teuthology.orchestra.run.smithi137.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-26T04:28:44.523 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-26T04:28:44.523 INFO:teuthology.orchestra.run.smithi137.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-26T04:28:44.541 INFO:teuthology.orchestra.run.smithi178.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-26T04:28:44.541 INFO:teuthology.orchestra.run.smithi178.stdout:Need to get 179 MB of archives. 2024-04-26T04:28:44.541 INFO:teuthology.orchestra.run.smithi178.stdout:After this operation, 822 MB of additional disk space will be used. 2024-04-26T04:28:44.541 INFO:teuthology.orchestra.run.smithi178.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-26T04:28:44.569 INFO:teuthology.orchestra.run.smithi113.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-26T04:28:44.569 INFO:teuthology.orchestra.run.smithi113.stdout:Need to get 179 MB of archives. 2024-04-26T04:28:44.570 INFO:teuthology.orchestra.run.smithi113.stdout:After this operation, 822 MB of additional disk space will be used. 2024-04-26T04:28:44.570 INFO:teuthology.orchestra.run.smithi113.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2457-gddeaa917-1jammy [3,545 kB] 2024-04-26T04:28:44.589 INFO:teuthology.orchestra.run.smithi137.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-26T04:28:44.589 INFO:teuthology.orchestra.run.smithi137.stdout:Need to get 179 MB of archives. 2024-04-26T04:28:44.589 INFO:teuthology.orchestra.run.smithi137.stdout:After this operation, 822 MB of additional disk space will be used. 2024-04-26T04:28:44.589 INFO:teuthology.orchestra.run.smithi137.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-26T04:28:44.617 INFO:teuthology.orchestra.run.smithi113.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-26T04:28:44.648 INFO:teuthology.orchestra.run.smithi178.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-26T04:28:44.650 INFO:teuthology.orchestra.run.smithi178.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-26T04:28:44.667 INFO:teuthology.orchestra.run.smithi012.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-26T04:28:44.667 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 179 MB of archives. 2024-04-26T04:28:44.667 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 822 MB of additional disk space will be used. 2024-04-26T04:28:44.667 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-26T04:28:44.675 INFO:teuthology.orchestra.run.smithi178.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-26T04:28:44.694 INFO:teuthology.orchestra.run.smithi178.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2457-gddeaa917-1jammy [3,545 kB] 2024-04-26T04:28:44.705 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2457-gddeaa917-1jammy [3,545 kB] 2024-04-26T04:28:44.710 INFO:teuthology.orchestra.run.smithi137.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-26T04:28:44.713 INFO:teuthology.orchestra.run.smithi137.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-26T04:28:44.738 INFO:teuthology.orchestra.run.smithi137.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-26T04:28:44.756 INFO:teuthology.orchestra.run.smithi178.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-26T04:28:44.761 INFO:teuthology.orchestra.run.smithi178.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-26T04:28:44.764 INFO:teuthology.orchestra.run.smithi137.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2457-gddeaa917-1jammy [3,545 kB] 2024-04-26T04:28:44.790 INFO:teuthology.orchestra.run.smithi178.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-26T04:28:44.797 INFO:teuthology.orchestra.run.smithi178.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-26T04:28:44.797 INFO:teuthology.orchestra.run.smithi178.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-26T04:28:44.798 INFO:teuthology.orchestra.run.smithi178.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-26T04:28:44.810 INFO:teuthology.orchestra.run.smithi178.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-26T04:28:44.813 INFO:teuthology.orchestra.run.smithi178.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-26T04:28:44.816 INFO:teuthology.orchestra.run.smithi178.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-26T04:28:44.817 INFO:teuthology.orchestra.run.smithi178.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-26T04:28:44.826 INFO:teuthology.orchestra.run.smithi178.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-26T04:28:44.829 INFO:teuthology.orchestra.run.smithi178.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-26T04:28:44.832 INFO:teuthology.orchestra.run.smithi178.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-26T04:28:44.834 INFO:teuthology.orchestra.run.smithi178.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-26T04:28:44.834 INFO:teuthology.orchestra.run.smithi178.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-26T04:28:44.848 INFO:teuthology.orchestra.run.smithi137.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-26T04:28:44.852 INFO:teuthology.orchestra.run.smithi178.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-26T04:28:44.854 INFO:teuthology.orchestra.run.smithi178.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-26T04:28:44.854 INFO:teuthology.orchestra.run.smithi178.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-26T04:28:44.854 INFO:teuthology.orchestra.run.smithi178.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-26T04:28:44.855 INFO:teuthology.orchestra.run.smithi178.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-26T04:28:44.857 INFO:teuthology.orchestra.run.smithi137.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-26T04:28:44.880 INFO:teuthology.orchestra.run.smithi137.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-26T04:28:44.883 INFO:teuthology.orchestra.run.smithi113.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2457-gddeaa917-1jammy [3,218 kB] 2024-04-26T04:28:44.897 INFO:teuthology.orchestra.run.smithi137.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-26T04:28:44.898 INFO:teuthology.orchestra.run.smithi137.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-26T04:28:44.899 INFO:teuthology.orchestra.run.smithi137.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-26T04:28:44.908 INFO:teuthology.orchestra.run.smithi178.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-26T04:28:44.908 INFO:teuthology.orchestra.run.smithi178.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-26T04:28:44.908 INFO:teuthology.orchestra.run.smithi178.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-26T04:28:44.913 INFO:teuthology.orchestra.run.smithi178.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-26T04:28:44.914 INFO:teuthology.orchestra.run.smithi178.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-26T04:28:44.915 INFO:teuthology.orchestra.run.smithi137.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-26T04:28:44.927 INFO:teuthology.orchestra.run.smithi137.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-26T04:28:44.927 INFO:teuthology.orchestra.run.smithi137.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-26T04:28:44.930 INFO:teuthology.orchestra.run.smithi137.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-26T04:28:44.935 INFO:teuthology.orchestra.run.smithi137.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-26T04:28:44.935 INFO:teuthology.orchestra.run.smithi178.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-26T04:28:44.935 INFO:teuthology.orchestra.run.smithi178.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-26T04:28:44.936 INFO:teuthology.orchestra.run.smithi137.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-26T04:28:44.945 INFO:teuthology.orchestra.run.smithi137.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-26T04:28:44.946 INFO:teuthology.orchestra.run.smithi137.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-26T04:28:44.946 INFO:teuthology.orchestra.run.smithi137.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-26T04:28:44.946 INFO:teuthology.orchestra.run.smithi178.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-26T04:28:44.946 INFO:teuthology.orchestra.run.smithi178.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-26T04:28:44.946 INFO:teuthology.orchestra.run.smithi178.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-26T04:28:44.959 INFO:teuthology.orchestra.run.smithi137.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-26T04:28:44.964 INFO:teuthology.orchestra.run.smithi178.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-26T04:28:44.964 INFO:teuthology.orchestra.run.smithi178.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-26T04:28:44.964 INFO:teuthology.orchestra.run.smithi178.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-26T04:28:44.964 INFO:teuthology.orchestra.run.smithi137.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-26T04:28:44.964 INFO:teuthology.orchestra.run.smithi137.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-26T04:28:44.964 INFO:teuthology.orchestra.run.smithi137.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-26T04:28:44.964 INFO:teuthology.orchestra.run.smithi137.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-26T04:28:44.974 INFO:teuthology.orchestra.run.smithi178.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-26T04:28:44.974 INFO:teuthology.orchestra.run.smithi178.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-26T04:28:44.984 INFO:teuthology.orchestra.run.smithi137.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-26T04:28:44.985 INFO:teuthology.orchestra.run.smithi137.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-26T04:28:44.985 INFO:teuthology.orchestra.run.smithi137.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-26T04:28:44.986 INFO:teuthology.orchestra.run.smithi178.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-26T04:28:44.988 INFO:teuthology.orchestra.run.smithi178.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-26T04:28:44.989 INFO:teuthology.orchestra.run.smithi137.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-26T04:28:44.990 INFO:teuthology.orchestra.run.smithi178.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-26T04:28:44.990 INFO:teuthology.orchestra.run.smithi137.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-26T04:28:44.993 INFO:teuthology.orchestra.run.smithi178.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-26T04:28:44.998 INFO:teuthology.orchestra.run.smithi178.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-26T04:28:45.008 INFO:teuthology.orchestra.run.smithi178.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-26T04:28:45.009 INFO:teuthology.orchestra.run.smithi137.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-26T04:28:45.009 INFO:teuthology.orchestra.run.smithi137.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-26T04:28:45.009 INFO:teuthology.orchestra.run.smithi137.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-26T04:28:45.009 INFO:teuthology.orchestra.run.smithi137.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-26T04:28:45.009 INFO:teuthology.orchestra.run.smithi137.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-26T04:28:45.009 INFO:teuthology.orchestra.run.smithi178.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-26T04:28:45.013 INFO:teuthology.orchestra.run.smithi178.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-26T04:28:45.013 INFO:teuthology.orchestra.run.smithi178.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-26T04:28:45.015 INFO:teuthology.orchestra.run.smithi178.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-26T04:28:45.028 INFO:teuthology.orchestra.run.smithi137.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-26T04:28:45.029 INFO:teuthology.orchestra.run.smithi137.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-26T04:28:45.030 INFO:teuthology.orchestra.run.smithi137.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-26T04:28:45.030 INFO:teuthology.orchestra.run.smithi113.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2457-gddeaa917-1jammy [711 kB] 2024-04-26T04:28:45.034 INFO:teuthology.orchestra.run.smithi178.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-26T04:28:45.043 INFO:teuthology.orchestra.run.smithi137.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-26T04:28:45.044 INFO:teuthology.orchestra.run.smithi137.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-26T04:28:45.046 INFO:teuthology.orchestra.run.smithi113.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-26T04:28:45.059 INFO:teuthology.orchestra.run.smithi113.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-26T04:28:45.065 INFO:teuthology.orchestra.run.smithi113.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2457-gddeaa917-1jammy [356 kB] 2024-04-26T04:28:45.081 INFO:teuthology.orchestra.run.smithi113.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2457-gddeaa917-1jammy [32.8 kB] 2024-04-26T04:28:45.081 INFO:teuthology.orchestra.run.smithi113.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2457-gddeaa917-1jammy [180 kB] 2024-04-26T04:28:45.084 INFO:teuthology.orchestra.run.smithi137.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-26T04:28:45.086 INFO:teuthology.orchestra.run.smithi113.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2457-gddeaa917-1jammy [65.8 kB] 2024-04-26T04:28:45.087 INFO:teuthology.orchestra.run.smithi137.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-26T04:28:45.087 INFO:teuthology.orchestra.run.smithi137.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-26T04:28:45.089 INFO:teuthology.orchestra.run.smithi113.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2457-gddeaa917-1jammy [330 kB] 2024-04-26T04:28:45.090 INFO:teuthology.orchestra.run.smithi137.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-26T04:28:45.091 INFO:teuthology.orchestra.run.smithi137.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-26T04:28:45.100 INFO:teuthology.orchestra.run.smithi113.stdout:Get:12 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2457-gddeaa917-1jammy [6,775 kB] 2024-04-26T04:28:45.133 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-26T04:28:45.137 INFO:teuthology.orchestra.run.smithi178.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-26T04:28:45.138 INFO:teuthology.orchestra.run.smithi137.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-26T04:28:45.139 INFO:teuthology.orchestra.run.smithi178.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-26T04:28:45.141 INFO:teuthology.orchestra.run.smithi113.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-26T04:28:45.141 INFO:teuthology.orchestra.run.smithi137.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-26T04:28:45.142 INFO:teuthology.orchestra.run.smithi178.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-26T04:28:45.142 INFO:teuthology.orchestra.run.smithi178.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-26T04:28:45.143 INFO:teuthology.orchestra.run.smithi178.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-26T04:28:45.144 INFO:teuthology.orchestra.run.smithi137.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-26T04:28:45.145 INFO:teuthology.orchestra.run.smithi137.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-26T04:28:45.147 INFO:teuthology.orchestra.run.smithi137.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-26T04:28:45.148 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-26T04:28:45.167 INFO:teuthology.orchestra.run.smithi178.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-26T04:28:45.169 INFO:teuthology.orchestra.run.smithi178.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-26T04:28:45.170 INFO:teuthology.orchestra.run.smithi178.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-26T04:28:45.174 INFO:teuthology.orchestra.run.smithi137.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-26T04:28:45.204 INFO:teuthology.orchestra.run.smithi178.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-26T04:28:45.204 INFO:teuthology.orchestra.run.smithi178.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-26T04:28:45.244 INFO:teuthology.orchestra.run.smithi178.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-26T04:28:45.244 INFO:teuthology.orchestra.run.smithi178.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-26T04:28:45.244 INFO:teuthology.orchestra.run.smithi178.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-26T04:28:45.245 INFO:teuthology.orchestra.run.smithi178.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-26T04:28:45.245 INFO:teuthology.orchestra.run.smithi178.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-26T04:28:45.246 INFO:teuthology.orchestra.run.smithi178.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-26T04:28:45.273 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-26T04:28:45.299 INFO:teuthology.orchestra.run.smithi178.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-26T04:28:45.312 INFO:teuthology.orchestra.run.smithi178.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-26T04:28:45.314 INFO:teuthology.orchestra.run.smithi178.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-26T04:28:45.315 INFO:teuthology.orchestra.run.smithi178.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-26T04:28:45.322 INFO:teuthology.orchestra.run.smithi178.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-26T04:28:45.327 INFO:teuthology.orchestra.run.smithi178.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-26T04:28:45.328 INFO:teuthology.orchestra.run.smithi178.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-26T04:28:45.329 INFO:teuthology.orchestra.run.smithi178.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-26T04:28:45.339 INFO:teuthology.orchestra.run.smithi137.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-26T04:28:45.340 INFO:teuthology.orchestra.run.smithi137.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-26T04:28:45.342 INFO:teuthology.orchestra.run.smithi137.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-26T04:28:45.343 INFO:teuthology.orchestra.run.smithi137.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-26T04:28:45.343 INFO:teuthology.orchestra.run.smithi178.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-26T04:28:45.344 INFO:teuthology.orchestra.run.smithi178.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-26T04:28:45.345 INFO:teuthology.orchestra.run.smithi137.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-26T04:28:45.368 INFO:teuthology.orchestra.run.smithi178.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-26T04:28:45.368 INFO:teuthology.orchestra.run.smithi178.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-26T04:28:45.369 INFO:teuthology.orchestra.run.smithi137.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-26T04:28:45.369 INFO:teuthology.orchestra.run.smithi178.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-26T04:28:45.370 INFO:teuthology.orchestra.run.smithi178.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-26T04:28:45.371 INFO:teuthology.orchestra.run.smithi137.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-26T04:28:45.371 INFO:teuthology.orchestra.run.smithi113.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-26T04:28:45.371 INFO:teuthology.orchestra.run.smithi137.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-26T04:28:45.373 INFO:teuthology.orchestra.run.smithi178.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-26T04:28:45.374 INFO:teuthology.orchestra.run.smithi178.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-26T04:28:45.383 INFO:teuthology.orchestra.run.smithi113.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-26T04:28:45.387 INFO:teuthology.orchestra.run.smithi178.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-26T04:28:45.393 INFO:teuthology.orchestra.run.smithi178.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-26T04:28:45.393 INFO:teuthology.orchestra.run.smithi178.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-26T04:28:45.395 INFO:teuthology.orchestra.run.smithi113.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2457-gddeaa917-1jammy [112 kB] 2024-04-26T04:28:45.397 INFO:teuthology.orchestra.run.smithi178.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-26T04:28:45.400 INFO:teuthology.orchestra.run.smithi137.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-26T04:28:45.400 INFO:teuthology.orchestra.run.smithi113.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2457-gddeaa917-1jammy [466 kB] 2024-04-26T04:28:45.401 INFO:teuthology.orchestra.run.smithi137.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-26T04:28:45.402 INFO:teuthology.orchestra.run.smithi137.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-26T04:28:45.406 INFO:teuthology.orchestra.run.smithi137.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-26T04:28:45.406 INFO:teuthology.orchestra.run.smithi137.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-26T04:28:45.407 INFO:teuthology.orchestra.run.smithi137.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-26T04:28:45.407 INFO:teuthology.orchestra.run.smithi137.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-26T04:28:45.412 INFO:teuthology.orchestra.run.smithi113.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-26T04:28:45.413 INFO:teuthology.orchestra.run.smithi178.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-26T04:28:45.417 INFO:teuthology.orchestra.run.smithi113.stdout:Get:19 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2457-gddeaa917-1jammy [25.6 MB] 2024-04-26T04:28:45.422 INFO:teuthology.orchestra.run.smithi113.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-26T04:28:45.423 INFO:teuthology.orchestra.run.smithi113.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-26T04:28:45.425 INFO:teuthology.orchestra.run.smithi113.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-26T04:28:45.428 INFO:teuthology.orchestra.run.smithi137.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-26T04:28:45.439 INFO:teuthology.orchestra.run.smithi137.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-26T04:28:45.444 INFO:teuthology.orchestra.run.smithi113.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-26T04:28:45.459 INFO:teuthology.orchestra.run.smithi137.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-26T04:28:45.463 INFO:teuthology.orchestra.run.smithi137.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-26T04:28:45.464 INFO:teuthology.orchestra.run.smithi137.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-26T04:28:45.473 INFO:teuthology.orchestra.run.smithi137.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-26T04:28:45.479 INFO:teuthology.orchestra.run.smithi137.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-26T04:28:45.481 INFO:teuthology.orchestra.run.smithi137.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-26T04:28:45.482 INFO:teuthology.orchestra.run.smithi137.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-26T04:28:45.491 INFO:teuthology.orchestra.run.smithi137.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-26T04:28:45.492 INFO:teuthology.orchestra.run.smithi137.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-26T04:28:45.498 INFO:teuthology.orchestra.run.smithi137.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-26T04:28:45.517 INFO:teuthology.orchestra.run.smithi137.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-26T04:28:45.519 INFO:teuthology.orchestra.run.smithi137.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-26T04:28:45.519 INFO:teuthology.orchestra.run.smithi137.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-26T04:28:45.522 INFO:teuthology.orchestra.run.smithi137.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-26T04:28:45.523 INFO:teuthology.orchestra.run.smithi137.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-26T04:28:45.542 INFO:teuthology.orchestra.run.smithi137.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-26T04:28:45.542 INFO:teuthology.orchestra.run.smithi137.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-26T04:28:45.542 INFO:teuthology.orchestra.run.smithi137.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-26T04:28:45.547 INFO:teuthology.orchestra.run.smithi137.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-26T04:28:45.548 INFO:teuthology.orchestra.run.smithi137.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-26T04:28:45.613 INFO:teuthology.orchestra.run.smithi113.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-26T04:28:45.702 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-26T04:28:45.723 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-26T04:28:45.756 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-26T04:28:45.767 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-26T04:28:45.768 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-26T04:28:45.769 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-26T04:28:45.789 INFO:teuthology.orchestra.run.smithi178.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2457-gddeaa917-1jammy [3,218 kB] 2024-04-26T04:28:45.792 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-26T04:28:45.962 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-26T04:28:46.023 INFO:teuthology.orchestra.run.smithi113.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-26T04:28:46.057 INFO:teuthology.orchestra.run.smithi113.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-26T04:28:46.059 INFO:teuthology.orchestra.run.smithi113.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-26T04:28:46.139 INFO:teuthology.orchestra.run.smithi113.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-26T04:28:46.190 INFO:teuthology.orchestra.run.smithi113.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-26T04:28:46.213 INFO:teuthology.orchestra.run.smithi113.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-26T04:28:46.222 INFO:teuthology.orchestra.run.smithi113.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-26T04:28:46.223 INFO:teuthology.orchestra.run.smithi113.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-26T04:28:46.232 INFO:teuthology.orchestra.run.smithi113.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-26T04:28:46.233 INFO:teuthology.orchestra.run.smithi113.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-26T04:28:46.277 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2457-gddeaa917-1jammy [3,218 kB] 2024-04-26T04:28:46.369 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-26T04:28:46.398 INFO:teuthology.orchestra.run.smithi113.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-26T04:28:46.402 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-26T04:28:46.404 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-26T04:28:46.481 INFO:teuthology.orchestra.run.smithi113.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-26T04:28:46.484 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-26T04:28:46.509 INFO:teuthology.orchestra.run.smithi113.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-26T04:28:46.526 INFO:teuthology.orchestra.run.smithi113.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-26T04:28:46.535 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-26T04:28:46.557 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-26T04:28:46.564 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-26T04:28:46.565 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-26T04:28:46.577 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-26T04:28:46.577 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-26T04:28:46.615 INFO:teuthology.orchestra.run.smithi113.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-26T04:28:46.643 INFO:teuthology.orchestra.run.smithi178.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2457-gddeaa917-1jammy [711 kB] 2024-04-26T04:28:46.681 INFO:teuthology.orchestra.run.smithi113.stdout:Get:40 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2457-gddeaa917-1jammy [5,084 kB] 2024-04-26T04:28:46.743 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-26T04:28:46.776 INFO:teuthology.orchestra.run.smithi178.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2457-gddeaa917-1jammy [356 kB] 2024-04-26T04:28:46.776 INFO:teuthology.orchestra.run.smithi113.stdout:Get:41 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2457-gddeaa917-1jammy [244 kB] 2024-04-26T04:28:46.786 INFO:teuthology.orchestra.run.smithi113.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-26T04:28:46.786 INFO:teuthology.orchestra.run.smithi113.stdout:Get:43 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2457-gddeaa917-1jammy [124 kB] 2024-04-26T04:28:46.793 INFO:teuthology.orchestra.run.smithi113.stdout:Get:44 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2457-gddeaa917-1jammy [1,527 kB] 2024-04-26T04:28:46.800 INFO:teuthology.orchestra.run.smithi113.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-26T04:28:46.807 INFO:teuthology.orchestra.run.smithi113.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-26T04:28:46.813 INFO:teuthology.orchestra.run.smithi113.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-26T04:28:46.826 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-26T04:28:46.838 INFO:teuthology.orchestra.run.smithi113.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-26T04:28:46.838 INFO:teuthology.orchestra.run.smithi113.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-26T04:28:46.851 INFO:teuthology.orchestra.run.smithi178.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2457-gddeaa917-1jammy [32.8 kB] 2024-04-26T04:28:46.854 INFO:teuthology.orchestra.run.smithi178.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2457-gddeaa917-1jammy [180 kB] 2024-04-26T04:28:46.856 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-26T04:28:46.864 INFO:teuthology.orchestra.run.smithi113.stdout:Get:50 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2457-gddeaa917-1jammy [6,245 kB] 2024-04-26T04:28:46.876 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-26T04:28:46.908 INFO:teuthology.orchestra.run.smithi178.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2457-gddeaa917-1jammy [65.8 kB] 2024-04-26T04:28:46.916 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-26T04:28:46.917 INFO:teuthology.orchestra.run.smithi178.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2457-gddeaa917-1jammy [330 kB] 2024-04-26T04:28:47.000 INFO:teuthology.orchestra.run.smithi178.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2457-gddeaa917-1jammy [6,775 kB] 2024-04-26T04:28:47.003 INFO:teuthology.orchestra.run.smithi113.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-26T04:28:47.049 INFO:teuthology.orchestra.run.smithi137.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2457-gddeaa917-1jammy [3,218 kB] 2024-04-26T04:28:47.147 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-26T04:28:47.159 INFO:teuthology.orchestra.run.smithi113.stdout:Get:52 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2457-gddeaa917-1jammy [22.7 MB] 2024-04-26T04:28:47.164 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-26T04:28:47.171 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-26T04:28:47.177 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-26T04:28:47.231 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-26T04:28:47.237 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-26T04:28:47.321 INFO:teuthology.orchestra.run.smithi113.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-26T04:28:47.350 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2457-gddeaa917-1jammy [711 kB] 2024-04-26T04:28:47.359 INFO:teuthology.orchestra.run.smithi113.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-26T04:28:47.371 INFO:teuthology.orchestra.run.smithi113.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-26T04:28:47.422 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-26T04:28:47.572 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2457-gddeaa917-1jammy [356 kB] 2024-04-26T04:28:47.575 INFO:teuthology.orchestra.run.smithi113.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-26T04:28:47.576 INFO:teuthology.orchestra.run.smithi113.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-26T04:28:47.581 INFO:teuthology.orchestra.run.smithi113.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-26T04:28:47.587 INFO:teuthology.orchestra.run.smithi113.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-26T04:28:47.603 INFO:teuthology.orchestra.run.smithi113.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-26T04:28:47.624 INFO:teuthology.orchestra.run.smithi113.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-26T04:28:47.630 INFO:teuthology.orchestra.run.smithi113.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-26T04:28:47.668 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2457-gddeaa917-1jammy [32.8 kB] 2024-04-26T04:28:47.669 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2457-gddeaa917-1jammy [180 kB] 2024-04-26T04:28:47.741 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2457-gddeaa917-1jammy [65.8 kB] 2024-04-26T04:28:47.745 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2457-gddeaa917-1jammy [330 kB] 2024-04-26T04:28:47.767 INFO:teuthology.orchestra.run.smithi012.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-26T04:28:47.805 INFO:teuthology.orchestra.run.smithi113.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-26T04:28:47.807 INFO:teuthology.orchestra.run.smithi012.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-26T04:28:47.818 INFO:teuthology.orchestra.run.smithi012.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-26T04:28:47.898 INFO:teuthology.orchestra.run.smithi012.stdout:Get:46 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2457-gddeaa917-1jammy [6,775 kB] 2024-04-26T04:28:48.068 INFO:teuthology.orchestra.run.smithi113.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-26T04:28:48.201 INFO:teuthology.orchestra.run.smithi012.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-26T04:28:48.201 INFO:teuthology.orchestra.run.smithi012.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-26T04:28:48.208 INFO:teuthology.orchestra.run.smithi012.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-26T04:28:48.212 INFO:teuthology.orchestra.run.smithi012.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-26T04:28:48.237 INFO:teuthology.orchestra.run.smithi012.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-26T04:28:48.285 INFO:teuthology.orchestra.run.smithi012.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-26T04:28:48.302 INFO:teuthology.orchestra.run.smithi113.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-26T04:28:48.303 INFO:teuthology.orchestra.run.smithi113.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-26T04:28:48.304 INFO:teuthology.orchestra.run.smithi012.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-26T04:28:48.304 INFO:teuthology.orchestra.run.smithi113.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2457-gddeaa917-1jammy [14.1 kB] 2024-04-26T04:28:48.305 INFO:teuthology.orchestra.run.smithi113.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-26T04:28:48.305 INFO:teuthology.orchestra.run.smithi113.stdout:Get:69 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2457-gddeaa917-1jammy [869 kB] 2024-04-26T04:28:48.322 INFO:teuthology.orchestra.run.smithi113.stdout:Get:70 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2457-gddeaa917-1jammy [2,438 kB] 2024-04-26T04:28:48.344 INFO:teuthology.orchestra.run.smithi113.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-26T04:28:48.372 INFO:teuthology.orchestra.run.smithi113.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2457-gddeaa917-1jammy [791 kB] 2024-04-26T04:28:48.409 INFO:teuthology.orchestra.run.smithi113.stdout:Get:73 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2457-gddeaa917-1jammy [147 kB] 2024-04-26T04:28:48.415 INFO:teuthology.orchestra.run.smithi113.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2457-gddeaa917-1jammy [2,293 kB] 2024-04-26T04:28:48.486 INFO:teuthology.orchestra.run.smithi012.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-26T04:28:48.512 INFO:teuthology.orchestra.run.smithi178.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2457-gddeaa917-1jammy [112 kB] 2024-04-26T04:28:48.516 INFO:teuthology.orchestra.run.smithi178.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2457-gddeaa917-1jammy [466 kB] 2024-04-26T04:28:48.527 INFO:teuthology.orchestra.run.smithi113.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2457-gddeaa917-1jammy [8,625 kB] 2024-04-26T04:28:48.654 INFO:teuthology.orchestra.run.smithi113.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-26T04:28:48.656 INFO:teuthology.orchestra.run.smithi113.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-26T04:28:48.658 INFO:teuthology.orchestra.run.smithi178.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2457-gddeaa917-1jammy [25.6 MB] 2024-04-26T04:28:48.658 INFO:teuthology.orchestra.run.smithi113.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-26T04:28:48.659 INFO:teuthology.orchestra.run.smithi113.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-26T04:28:48.661 INFO:teuthology.orchestra.run.smithi113.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-26T04:28:48.758 INFO:teuthology.orchestra.run.smithi012.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-26T04:28:48.883 INFO:teuthology.orchestra.run.smithi113.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-26T04:28:48.927 INFO:teuthology.orchestra.run.smithi113.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2457-gddeaa917-1jammy [14.2 kB] 2024-04-26T04:28:48.927 INFO:teuthology.orchestra.run.smithi113.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2457-gddeaa917-1jammy [50.7 MB] 2024-04-26T04:28:48.937 INFO:teuthology.orchestra.run.smithi012.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-26T04:28:48.951 INFO:teuthology.orchestra.run.smithi012.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-26T04:28:48.963 INFO:teuthology.orchestra.run.smithi137.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2457-gddeaa917-1jammy [711 kB] 2024-04-26T04:28:48.971 INFO:teuthology.orchestra.run.smithi012.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-26T04:28:49.038 INFO:teuthology.orchestra.run.smithi012.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-26T04:28:49.318 INFO:teuthology.orchestra.run.smithi113.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-26T04:28:49.323 INFO:teuthology.orchestra.run.smithi113.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-26T04:28:49.328 INFO:teuthology.orchestra.run.smithi012.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-26T04:28:49.330 INFO:teuthology.orchestra.run.smithi012.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-26T04:28:49.333 INFO:teuthology.orchestra.run.smithi012.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-26T04:28:49.334 INFO:teuthology.orchestra.run.smithi012.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-26T04:28:49.336 INFO:teuthology.orchestra.run.smithi012.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-26T04:28:49.453 INFO:teuthology.orchestra.run.smithi137.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2457-gddeaa917-1jammy [356 kB] 2024-04-26T04:28:49.555 INFO:teuthology.orchestra.run.smithi012.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-26T04:28:49.614 INFO:teuthology.orchestra.run.smithi113.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-26T04:28:49.615 INFO:teuthology.orchestra.run.smithi113.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-26T04:28:49.616 INFO:teuthology.orchestra.run.smithi137.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2457-gddeaa917-1jammy [32.8 kB] 2024-04-26T04:28:49.617 INFO:teuthology.orchestra.run.smithi113.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-26T04:28:49.618 INFO:teuthology.orchestra.run.smithi113.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-26T04:28:49.620 INFO:teuthology.orchestra.run.smithi113.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-26T04:28:49.623 INFO:teuthology.orchestra.run.smithi113.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-26T04:28:49.623 INFO:teuthology.orchestra.run.smithi113.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-26T04:28:49.625 INFO:teuthology.orchestra.run.smithi113.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-26T04:28:49.664 INFO:teuthology.orchestra.run.smithi137.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2457-gddeaa917-1jammy [180 kB] 2024-04-26T04:28:49.744 INFO:teuthology.orchestra.run.smithi137.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2457-gddeaa917-1jammy [65.8 kB] 2024-04-26T04:28:49.750 INFO:teuthology.orchestra.run.smithi137.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2457-gddeaa917-1jammy [330 kB] 2024-04-26T04:28:49.808 INFO:teuthology.orchestra.run.smithi113.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-26T04:28:49.968 INFO:teuthology.orchestra.run.smithi137.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2457-gddeaa917-1jammy [6,775 kB] 2024-04-26T04:28:50.037 INFO:teuthology.orchestra.run.smithi012.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2457-gddeaa917-1jammy [112 kB] 2024-04-26T04:28:50.041 INFO:teuthology.orchestra.run.smithi012.stdout:Get:67 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2457-gddeaa917-1jammy [466 kB] 2024-04-26T04:28:50.171 INFO:teuthology.orchestra.run.smithi012.stdout:Get:68 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2457-gddeaa917-1jammy [25.6 MB] 2024-04-26T04:28:50.223 INFO:teuthology.orchestra.run.smithi113.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-26T04:28:50.269 INFO:teuthology.orchestra.run.smithi113.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-26T04:28:50.304 INFO:teuthology.orchestra.run.smithi113.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-26T04:28:50.401 INFO:teuthology.orchestra.run.smithi113.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-26T04:28:50.441 INFO:teuthology.orchestra.run.smithi113.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-26T04:28:50.452 INFO:teuthology.orchestra.run.smithi113.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-26T04:28:50.458 INFO:teuthology.orchestra.run.smithi113.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-26T04:28:50.504 INFO:teuthology.orchestra.run.smithi113.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-26T04:28:50.508 INFO:teuthology.orchestra.run.smithi113.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-26T04:28:50.527 INFO:teuthology.orchestra.run.smithi012.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-26T04:28:50.529 INFO:teuthology.orchestra.run.smithi113.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-26T04:28:50.583 INFO:teuthology.orchestra.run.smithi012.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-26T04:28:50.693 INFO:teuthology.orchestra.run.smithi113.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-26T04:28:50.957 INFO:teuthology.orchestra.run.smithi113.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-26T04:28:50.976 INFO:teuthology.orchestra.run.smithi113.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-26T04:28:51.132 INFO:teuthology.orchestra.run.smithi113.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-26T04:28:51.156 INFO:teuthology.orchestra.run.smithi113.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-26T04:28:51.313 INFO:teuthology.orchestra.run.smithi113.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-26T04:28:51.316 INFO:teuthology.orchestra.run.smithi113.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-26T04:28:51.318 INFO:teuthology.orchestra.run.smithi113.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-26T04:28:51.340 INFO:teuthology.orchestra.run.smithi113.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-26T04:28:51.345 INFO:teuthology.orchestra.run.smithi113.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-26T04:28:51.582 INFO:teuthology.orchestra.run.smithi113.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2457-gddeaa917-1jammy [121 kB] 2024-04-26T04:28:51.584 INFO:teuthology.orchestra.run.smithi113.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2457-gddeaa917-1jammy [39.0 kB] 2024-04-26T04:28:51.585 INFO:teuthology.orchestra.run.smithi113.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2457-gddeaa917-1jammy [13.4 MB] 2024-04-26T04:28:52.121 INFO:teuthology.orchestra.run.smithi113.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2457-gddeaa917-1jammy [91.2 kB] 2024-04-26T04:28:52.670 INFO:teuthology.orchestra.run.smithi113.stdout:Fetched 179 MB in 8s (23.4 MB/s) 2024-04-26T04:28:52.672 INFO:teuthology.orchestra.run.smithi137.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2457-gddeaa917-1jammy [112 kB] 2024-04-26T04:28:52.706 INFO:teuthology.orchestra.run.smithi137.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2457-gddeaa917-1jammy [466 kB] 2024-04-26T04:28:52.800 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-26T04:28:52.830 INFO:teuthology.orchestra.run.smithi137.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2457-gddeaa917-1jammy [25.6 MB] 2024-04-26T04:28:52.898 INFO:teuthology.orchestra.run.smithi113.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-26T04:28:52.903 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-26T04:28:52.990 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-26T04:28:53.319 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-26T04:28:53.345 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-26T04:28:53.385 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-26T04:28:53.417 INFO:teuthology.orchestra.run.smithi178.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2457-gddeaa917-1jammy [5,084 kB] 2024-04-26T04:28:53.663 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-26T04:28:53.690 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-26T04:28:53.730 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-26T04:28:53.781 INFO:teuthology.orchestra.run.smithi012.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-26T04:28:53.789 INFO:teuthology.orchestra.run.smithi012.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-26T04:28:53.847 INFO:teuthology.orchestra.run.smithi012.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-26T04:28:53.875 INFO:teuthology.orchestra.run.smithi012.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-26T04:28:53.918 INFO:teuthology.orchestra.run.smithi012.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-26T04:28:54.025 INFO:teuthology.orchestra.run.smithi012.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-26T04:28:54.038 INFO:teuthology.orchestra.run.smithi012.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-26T04:28:54.071 INFO:teuthology.orchestra.run.smithi178.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2457-gddeaa917-1jammy [244 kB] 2024-04-26T04:28:54.083 INFO:teuthology.orchestra.run.smithi178.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2457-gddeaa917-1jammy [124 kB] 2024-04-26T04:28:54.121 INFO:teuthology.orchestra.run.smithi178.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2457-gddeaa917-1jammy [1,527 kB] 2024-04-26T04:28:54.154 INFO:teuthology.orchestra.run.smithi012.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-26T04:28:54.289 INFO:teuthology.orchestra.run.smithi178.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2457-gddeaa917-1jammy [6,245 kB] 2024-04-26T04:28:54.317 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-26T04:28:54.343 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-26T04:28:54.392 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:28:54.745 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-26T04:28:54.772 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-26T04:28:54.803 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:28:54.872 INFO:teuthology.orchestra.run.smithi012.stdout:Get:79 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2457-gddeaa917-1jammy [5,084 kB] 2024-04-26T04:28:55.037 INFO:teuthology.orchestra.run.smithi012.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-26T04:28:55.047 INFO:teuthology.orchestra.run.smithi178.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2457-gddeaa917-1jammy [22.7 MB] 2024-04-26T04:28:55.139 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-26T04:28:55.166 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-26T04:28:55.206 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:28:55.443 INFO:teuthology.orchestra.run.smithi012.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-26T04:28:55.473 INFO:teuthology.orchestra.run.smithi012.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2457-gddeaa917-1jammy [244 kB] 2024-04-26T04:28:55.505 INFO:teuthology.orchestra.run.smithi012.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2457-gddeaa917-1jammy [124 kB] 2024-04-26T04:28:55.509 INFO:teuthology.orchestra.run.smithi012.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2457-gddeaa917-1jammy [1,527 kB] 2024-04-26T04:28:55.526 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-26T04:28:55.546 INFO:teuthology.orchestra.run.smithi012.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-26T04:28:55.553 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-26T04:28:55.573 INFO:teuthology.orchestra.run.smithi012.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-26T04:28:55.585 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-26T04:28:55.709 INFO:teuthology.orchestra.run.smithi012.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2457-gddeaa917-1jammy [6,245 kB] 2024-04-26T04:28:55.739 INFO:teuthology.orchestra.run.smithi012.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-26T04:28:55.809 INFO:teuthology.orchestra.run.smithi012.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-26T04:28:55.824 INFO:teuthology.orchestra.run.smithi012.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-26T04:28:55.890 INFO:teuthology.orchestra.run.smithi012.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-26T04:28:55.904 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package librados2. 2024-04-26T04:28:55.931 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../007-librados2_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:28:55.970 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking librados2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:28:56.166 INFO:teuthology.orchestra.run.smithi012.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-26T04:28:56.206 INFO:teuthology.orchestra.run.smithi012.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-26T04:28:56.432 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package librbd1. 2024-04-26T04:28:56.458 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../008-librbd1_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:28:56.465 INFO:teuthology.orchestra.run.smithi012.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-26T04:28:56.467 INFO:teuthology.orchestra.run.smithi012.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2457-gddeaa917-1jammy [22.7 MB] 2024-04-26T04:28:56.498 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking librbd1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:28:56.634 INFO:teuthology.orchestra.run.smithi012.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-26T04:28:56.909 INFO:teuthology.orchestra.run.smithi012.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-26T04:28:56.927 INFO:teuthology.orchestra.run.smithi012.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-26T04:28:56.943 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libcephfs2. 2024-04-26T04:28:56.969 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:28:57.009 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libcephfs2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:28:57.082 INFO:teuthology.orchestra.run.smithi012.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-26T04:28:57.111 INFO:teuthology.orchestra.run.smithi012.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-26T04:28:57.269 INFO:teuthology.orchestra.run.smithi012.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-26T04:28:57.270 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-rados. 2024-04-26T04:28:57.271 INFO:teuthology.orchestra.run.smithi012.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-26T04:28:57.273 INFO:teuthology.orchestra.run.smithi012.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-26T04:28:57.297 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:28:57.297 INFO:teuthology.orchestra.run.smithi012.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-26T04:28:57.301 INFO:teuthology.orchestra.run.smithi012.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-26T04:28:57.328 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-rados (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:28:57.384 INFO:teuthology.orchestra.run.smithi137.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2457-gddeaa917-1jammy [5,084 kB] 2024-04-26T04:28:57.590 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-26T04:28:57.616 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:28:57.652 INFO:teuthology.orchestra.run.smithi178.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2457-gddeaa917-1jammy [14.1 kB] 2024-04-26T04:28:57.653 INFO:teuthology.orchestra.run.smithi178.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2457-gddeaa917-1jammy [869 kB] 2024-04-26T04:28:57.656 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-ceph-argparse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:28:57.669 INFO:teuthology.orchestra.run.smithi178.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2457-gddeaa917-1jammy [2,438 kB] 2024-04-26T04:28:57.842 INFO:teuthology.orchestra.run.smithi137.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2457-gddeaa917-1jammy [244 kB] 2024-04-26T04:28:57.860 INFO:teuthology.orchestra.run.smithi137.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2457-gddeaa917-1jammy [124 kB] 2024-04-26T04:28:57.893 INFO:teuthology.orchestra.run.smithi137.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2457-gddeaa917-1jammy [1,527 kB] 2024-04-26T04:28:57.909 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cephfs. 2024-04-26T04:28:57.934 INFO:teuthology.orchestra.run.smithi178.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2457-gddeaa917-1jammy [791 kB] 2024-04-26T04:28:57.935 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:28:57.975 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cephfs (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:28:58.020 INFO:teuthology.orchestra.run.smithi178.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2457-gddeaa917-1jammy [147 kB] 2024-04-26T04:28:58.047 INFO:teuthology.orchestra.run.smithi178.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2457-gddeaa917-1jammy [2,293 kB] 2024-04-26T04:28:58.061 INFO:teuthology.orchestra.run.smithi137.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2457-gddeaa917-1jammy [6,245 kB] 2024-04-26T04:28:58.266 INFO:teuthology.orchestra.run.smithi178.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2457-gddeaa917-1jammy [8,625 kB] 2024-04-26T04:28:58.286 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-26T04:28:58.313 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:28:58.353 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:28:58.643 INFO:teuthology.orchestra.run.smithi137.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2457-gddeaa917-1jammy [22.7 MB] 2024-04-26T04:28:58.681 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-26T04:28:58.707 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-26T04:28:58.747 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-26T04:28:59.017 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-prettytable. 2024-04-26T04:28:59.032 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-26T04:28:59.068 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-26T04:28:59.168 INFO:teuthology.orchestra.run.smithi012.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2457-gddeaa917-1jammy [14.1 kB] 2024-04-26T04:28:59.168 INFO:teuthology.orchestra.run.smithi012.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2457-gddeaa917-1jammy [869 kB] 2024-04-26T04:28:59.254 INFO:teuthology.orchestra.run.smithi012.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2457-gddeaa917-1jammy [2,438 kB] 2024-04-26T04:28:59.438 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-rbd. 2024-04-26T04:28:59.452 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:28:59.486 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-rbd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:28:59.492 INFO:teuthology.orchestra.run.smithi178.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2457-gddeaa917-1jammy [14.2 kB] 2024-04-26T04:28:59.492 INFO:teuthology.orchestra.run.smithi178.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2457-gddeaa917-1jammy [50.7 MB] 2024-04-26T04:28:59.549 INFO:teuthology.orchestra.run.smithi012.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2457-gddeaa917-1jammy [791 kB] 2024-04-26T04:28:59.634 INFO:teuthology.orchestra.run.smithi012.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2457-gddeaa917-1jammy [147 kB] 2024-04-26T04:28:59.647 INFO:teuthology.orchestra.run.smithi012.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2457-gddeaa917-1jammy [2,293 kB] 2024-04-26T04:28:59.764 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-26T04:28:59.777 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-26T04:28:59.806 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-26T04:28:59.877 INFO:teuthology.orchestra.run.smithi012.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2457-gddeaa917-1jammy [8,625 kB] 2024-04-26T04:29:00.042 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-26T04:29:00.057 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-26T04:29:00.092 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-26T04:29:00.369 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-26T04:29:00.385 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-26T04:29:00.428 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-26T04:29:00.740 INFO:teuthology.orchestra.run.smithi012.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2457-gddeaa917-1jammy [14.2 kB] 2024-04-26T04:29:00.741 INFO:teuthology.orchestra.run.smithi012.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2457-gddeaa917-1jammy [50.7 MB] 2024-04-26T04:29:00.781 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package lua5.1. 2024-04-26T04:29:00.807 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-26T04:29:00.847 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-26T04:29:00.878 INFO:teuthology.orchestra.run.smithi137.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2457-gddeaa917-1jammy [14.1 kB] 2024-04-26T04:29:00.879 INFO:teuthology.orchestra.run.smithi137.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2457-gddeaa917-1jammy [869 kB] 2024-04-26T04:29:00.927 INFO:teuthology.orchestra.run.smithi137.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2457-gddeaa917-1jammy [2,438 kB] 2024-04-26T04:29:01.092 INFO:teuthology.orchestra.run.smithi137.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2457-gddeaa917-1jammy [791 kB] 2024-04-26T04:29:01.142 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package lua-any. 2024-04-26T04:29:01.158 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-26T04:29:01.170 INFO:teuthology.orchestra.run.smithi137.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2457-gddeaa917-1jammy [147 kB] 2024-04-26T04:29:01.176 INFO:teuthology.orchestra.run.smithi137.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2457-gddeaa917-1jammy [2,293 kB] 2024-04-26T04:29:01.192 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-26T04:29:01.363 INFO:teuthology.orchestra.run.smithi137.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2457-gddeaa917-1jammy [8,625 kB] 2024-04-26T04:29:01.394 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package zip. 2024-04-26T04:29:01.409 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-26T04:29:01.453 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking zip (3.0-12build2) ... 2024-04-26T04:29:01.764 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package unzip. 2024-04-26T04:29:01.791 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-26T04:29:01.831 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-26T04:29:02.135 INFO:teuthology.orchestra.run.smithi137.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2457-gddeaa917-1jammy [14.2 kB] 2024-04-26T04:29:02.136 INFO:teuthology.orchestra.run.smithi137.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2457-gddeaa917-1jammy [50.7 MB] 2024-04-26T04:29:02.219 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package luarocks. 2024-04-26T04:29:02.245 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-26T04:29:02.285 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-26T04:29:02.643 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package librgw2. 2024-04-26T04:29:02.669 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../025-librgw2_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:02.701 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking librgw2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:03.221 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-rgw. 2024-04-26T04:29:03.235 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:03.270 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-rgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:03.598 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-26T04:29:03.624 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-26T04:29:03.664 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-26T04:29:03.976 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libradosstriper1. 2024-04-26T04:29:04.003 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:04.042 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libradosstriper1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:04.396 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-common. 2024-04-26T04:29:04.422 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:04.504 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:05.776 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-base. 2024-04-26T04:29:05.802 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:05.803 INFO:teuthology.orchestra.run.smithi178.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2457-gddeaa917-1jammy [121 kB] 2024-04-26T04:29:05.806 INFO:teuthology.orchestra.run.smithi178.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2457-gddeaa917-1jammy [39.0 kB] 2024-04-26T04:29:05.807 INFO:teuthology.orchestra.run.smithi178.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2457-gddeaa917-1jammy [13.4 MB] 2024-04-26T04:29:05.852 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-base (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:06.063 INFO:teuthology.orchestra.run.smithi012.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2457-gddeaa917-1jammy [121 kB] 2024-04-26T04:29:06.065 INFO:teuthology.orchestra.run.smithi012.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2457-gddeaa917-1jammy [39.0 kB] 2024-04-26T04:29:06.066 INFO:teuthology.orchestra.run.smithi012.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2457-gddeaa917-1jammy [13.4 MB] 2024-04-26T04:29:06.413 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-26T04:29:06.439 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-26T04:29:06.480 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-26T04:29:06.601 INFO:teuthology.orchestra.run.smithi137.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2457-gddeaa917-1jammy [121 kB] 2024-04-26T04:29:06.604 INFO:teuthology.orchestra.run.smithi137.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2457-gddeaa917-1jammy [39.0 kB] 2024-04-26T04:29:06.605 INFO:teuthology.orchestra.run.smithi137.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2457-gddeaa917-1jammy [13.4 MB] 2024-04-26T04:29:06.783 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cheroot. 2024-04-26T04:29:06.809 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-26T04:29:06.849 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-26T04:29:07.103 INFO:teuthology.orchestra.run.smithi178.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2457-gddeaa917-1jammy [91.2 kB] 2024-04-26T04:29:07.169 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-26T04:29:07.193 INFO:teuthology.orchestra.run.smithi012.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2457-gddeaa917-1jammy [91.2 kB] 2024-04-26T04:29:07.195 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-26T04:29:07.235 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-26T04:29:07.484 INFO:teuthology.orchestra.run.smithi137.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-yuri2-testing-2024-04-24-0914-squid/ddeaa91797339a19172c3036ff48cca58c12f448/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2457-gddeaa917-1jammy [91.2 kB] 2024-04-26T04:29:07.496 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-26T04:29:07.512 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-26T04:29:07.546 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-26T04:29:07.669 INFO:teuthology.orchestra.run.smithi178.stdout:Fetched 179 MB in 23s (7,929 kB/s) 2024-04-26T04:29:07.788 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 179 MB in 23s (7,904 kB/s) 2024-04-26T04:29:07.801 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-26T04:29:07.835 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-26T04:29:07.861 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-26T04:29:07.899 INFO:teuthology.orchestra.run.smithi178.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-26T04:29:07.901 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-26T04:29:07.904 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-26T04:29:07.951 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-26T04:29:07.982 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-26T04:29:08.045 INFO:teuthology.orchestra.run.smithi137.stdout:Fetched 179 MB in 23s (7,818 kB/s) 2024-04-26T04:29:08.053 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-26T04:29:08.058 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-26T04:29:08.141 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-26T04:29:08.174 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-26T04:29:08.188 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-tempora. 2024-04-26T04:29:08.214 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-26T04:29:08.254 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-26T04:29:08.271 INFO:teuthology.orchestra.run.smithi137.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-26T04:29:08.272 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-26T04:29:08.303 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-26T04:29:08.329 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-26T04:29:08.356 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-26T04:29:08.369 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-26T04:29:08.469 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-26T04:29:08.496 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-26T04:29:08.536 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-26T04:29:08.549 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-portend. 2024-04-26T04:29:08.575 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-26T04:29:08.615 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-26T04:29:08.693 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-26T04:29:08.705 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-26T04:29:08.719 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-26T04:29:08.731 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-26T04:29:08.759 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-26T04:29:08.772 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-26T04:29:08.847 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-26T04:29:08.874 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-26T04:29:08.918 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-26T04:29:08.945 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-26T04:29:08.985 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-26T04:29:09.039 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-26T04:29:09.095 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-26T04:29:09.109 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-26T04:29:09.111 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-26T04:29:09.135 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-26T04:29:09.154 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-26T04:29:09.175 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:29:09.271 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-26T04:29:09.298 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-26T04:29:09.338 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-26T04:29:09.443 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-26T04:29:09.470 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-26T04:29:09.518 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:29:09.566 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-26T04:29:09.578 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-26T04:29:09.592 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-26T04:29:09.604 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-26T04:29:09.632 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:29:09.645 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:29:09.858 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-natsort. 2024-04-26T04:29:09.884 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-26T04:29:09.924 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-26T04:29:09.931 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-26T04:29:09.937 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-26T04:29:09.958 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-26T04:29:09.964 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-26T04:29:10.004 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:29:10.014 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:29:10.019 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-26T04:29:10.034 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-26T04:29:10.077 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:29:10.236 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-logutils. 2024-04-26T04:29:10.263 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-26T04:29:10.303 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-26T04:29:10.307 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-26T04:29:10.334 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-26T04:29:10.365 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:29:10.367 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-26T04:29:10.394 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-26T04:29:10.430 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-26T04:29:10.434 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-26T04:29:10.457 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-26T04:29:10.497 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:29:10.555 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-26T04:29:10.571 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-26T04:29:10.606 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-26T04:29:10.745 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package librados2. 2024-04-26T04:29:10.771 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../007-librados2_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:10.812 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking librados2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:10.825 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-26T04:29:10.851 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-26T04:29:10.883 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-26T04:29:10.908 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-mako. 2024-04-26T04:29:10.935 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-26T04:29:10.975 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-26T04:29:10.994 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-26T04:29:11.020 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-26T04:29:11.060 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-26T04:29:11.320 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-26T04:29:11.347 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-26T04:29:11.352 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package librados2. 2024-04-26T04:29:11.379 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../007-librados2_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:11.380 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librados2. 2024-04-26T04:29:11.381 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package librbd1. 2024-04-26T04:29:11.387 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-26T04:29:11.397 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../008-librbd1_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:11.407 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../007-librados2_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:11.419 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking librados2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:11.431 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking librbd1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:11.446 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librados2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:11.690 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-26T04:29:11.716 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-26T04:29:11.756 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-26T04:29:11.874 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librbd1. 2024-04-26T04:29:11.890 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../008-librbd1_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:11.892 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libcephfs2. 2024-04-26T04:29:11.894 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package librbd1. 2024-04-26T04:29:11.919 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:11.921 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../008-librbd1_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:11.924 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librbd1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:11.959 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libcephfs2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:11.963 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking librbd1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:12.009 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-webob. 2024-04-26T04:29:12.032 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-26T04:29:12.067 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-26T04:29:12.253 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-rados. 2024-04-26T04:29:12.269 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:12.311 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-rados (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:12.352 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs2. 2024-04-26T04:29:12.379 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:12.404 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-waitress. 2024-04-26T04:29:12.408 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libcephfs2. 2024-04-26T04:29:12.419 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:12.430 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-26T04:29:12.435 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:12.473 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-26T04:29:12.474 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libcephfs2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:12.650 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-26T04:29:12.675 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:12.696 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rados. 2024-04-26T04:29:12.714 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-ceph-argparse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:12.723 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:12.763 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rados (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:12.782 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-tempita. 2024-04-26T04:29:12.808 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-26T04:29:12.835 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-rados. 2024-04-26T04:29:12.848 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-26T04:29:12.850 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:12.885 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-rados (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:12.941 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-cephfs. 2024-04-26T04:29:12.957 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:12.991 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-cephfs (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:13.082 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-26T04:29:13.109 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:13.135 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-paste. 2024-04-26T04:29:13.149 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-argparse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:13.161 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-26T04:29:13.205 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-26T04:29:13.209 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-26T04:29:13.229 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:13.269 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-26T04:29:13.271 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-ceph-argparse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:13.296 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:13.336 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:13.403 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cephfs. 2024-04-26T04:29:13.419 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:13.462 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cephfs (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:13.524 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-cephfs. 2024-04-26T04:29:13.529 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-26T04:29:13.545 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-26T04:29:13.550 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:13.588 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-26T04:29:13.590 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-cephfs (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:13.681 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-26T04:29:13.707 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-26T04:29:13.747 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-26T04:29:13.773 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-26T04:29:13.800 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:13.839 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:13.891 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-26T04:29:13.901 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-26T04:29:13.917 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-26T04:29:13.928 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:13.957 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-26T04:29:13.976 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:14.041 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-prettytable. 2024-04-26T04:29:14.068 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-26T04:29:14.108 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-26T04:29:14.167 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-26T04:29:14.194 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-26T04:29:14.234 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-26T04:29:14.235 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-webtest. 2024-04-26T04:29:14.262 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-26T04:29:14.271 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-26T04:29:14.294 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-26T04:29:14.297 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-26T04:29:14.329 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-26T04:29:14.344 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-rbd. 2024-04-26T04:29:14.359 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:14.394 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-rbd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:14.554 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-prettytable. 2024-04-26T04:29:14.580 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-26T04:29:14.605 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pecan. 2024-04-26T04:29:14.612 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-26T04:29:14.632 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-26T04:29:14.632 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-prettytable. 2024-04-26T04:29:14.659 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-26T04:29:14.672 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-26T04:29:14.699 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-26T04:29:14.739 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-26T04:29:14.765 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-26T04:29:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-26T04:29:14.848 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rbd. 2024-04-26T04:29:14.864 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:14.906 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rbd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:14.960 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-rbd. 2024-04-26T04:29:14.986 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:15.026 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-rbd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:15.027 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-certifi. 2024-04-26T04:29:15.049 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-26T04:29:15.053 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-26T04:29:15.064 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-26T04:29:15.093 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-26T04:29:15.099 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-26T04:29:15.250 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-26T04:29:15.277 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-26T04:29:15.317 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-26T04:29:15.362 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-26T04:29:15.369 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-26T04:29:15.389 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-26T04:29:15.395 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-26T04:29:15.396 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-idna. 2024-04-26T04:29:15.423 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-26T04:29:15.429 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-26T04:29:15.435 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-26T04:29:15.463 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-26T04:29:15.603 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-26T04:29:15.630 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-26T04:29:15.670 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-26T04:29:15.690 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-26T04:29:15.705 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-26T04:29:15.766 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-urllib3. 2024-04-26T04:29:15.772 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package lua5.1. 2024-04-26T04:29:15.793 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-26T04:29:15.798 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-26T04:29:15.832 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-26T04:29:15.838 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-26T04:29:16.207 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-26T04:29:16.258 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package lua-any. 2024-04-26T04:29:16.284 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-26T04:29:16.316 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-26T04:29:16.527 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package zip. 2024-04-26T04:29:16.554 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-26T04:29:16.594 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking zip (3.0-12build2) ... 2024-04-26T04:29:16.623 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-26T04:29:16.639 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-26T04:29:16.673 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-26T04:29:16.905 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package unzip. 2024-04-26T04:29:16.909 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-26T04:29:16.925 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-26T04:29:16.932 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-26T04:29:16.936 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-requests. 2024-04-26T04:29:16.963 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-26T04:29:16.967 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-26T04:29:16.972 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-26T04:29:16.976 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua5.1. 2024-04-26T04:29:17.002 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-26T04:29:17.004 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-26T04:29:17.043 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-26T04:29:17.296 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package lua5.1. 2024-04-26T04:29:17.314 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-26T04:29:17.323 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-26T04:29:17.341 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-26T04:29:17.354 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua-any. 2024-04-26T04:29:17.362 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-26T04:29:17.380 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-26T04:29:17.381 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-26T04:29:17.392 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package luarocks. 2024-04-26T04:29:17.419 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-26T04:29:17.421 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-26T04:29:17.459 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-26T04:29:17.640 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package zip. 2024-04-26T04:29:17.657 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package lua-any. 2024-04-26T04:29:17.667 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-26T04:29:17.672 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-26T04:29:17.675 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-26T04:29:17.702 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:17.707 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking zip (3.0-12build2) ... 2024-04-26T04:29:17.707 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-26T04:29:17.741 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:17.927 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package zip. 2024-04-26T04:29:17.942 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-26T04:29:17.985 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking zip (3.0-12build2) ... 2024-04-26T04:29:18.010 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package unzip. 2024-04-26T04:29:18.037 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-26T04:29:18.076 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-26T04:29:18.088 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-26T04:29:18.115 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:18.147 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:18.305 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package unzip. 2024-04-26T04:29:18.331 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-26T04:29:18.371 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-26T04:29:18.458 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-26T04:29:18.472 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package luarocks. 2024-04-26T04:29:18.485 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-26T04:29:18.499 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-26T04:29:18.524 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-26T04:29:18.538 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-26T04:29:18.618 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package librgw2. 2024-04-26T04:29:18.644 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../025-librgw2_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:18.684 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking librgw2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:18.741 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package luarocks. 2024-04-26T04:29:18.768 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-26T04:29:18.809 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-26T04:29:18.836 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-openssl. 2024-04-26T04:29:18.863 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-26T04:29:18.888 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librgw2. 2024-04-26T04:29:18.902 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-26T04:29:18.904 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../025-librgw2_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:18.938 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librgw2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:19.157 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package librgw2. 2024-04-26T04:29:19.184 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../025-librgw2_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:19.214 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr. 2024-04-26T04:29:19.220 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-rgw. 2024-04-26T04:29:19.224 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking librgw2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:19.235 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:19.240 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:19.270 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-rgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:19.272 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:19.490 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rgw. 2024-04-26T04:29:19.507 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:19.548 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:19.581 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-26T04:29:19.610 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-26T04:29:19.648 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-26T04:29:19.683 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mon. 2024-04-26T04:29:19.710 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:19.750 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mon (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:19.784 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-rgw. 2024-04-26T04:29:19.800 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:19.844 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-rgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:19.893 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-26T04:29:19.920 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-26T04:29:19.951 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libradosstriper1. 2024-04-26T04:29:19.959 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-26T04:29:19.977 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:20.018 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libradosstriper1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:20.162 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-26T04:29:20.189 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-26T04:29:20.229 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-26T04:29:20.254 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libradosstriper1. 2024-04-26T04:29:20.281 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:20.321 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libradosstriper1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:20.380 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-osd. 2024-04-26T04:29:20.395 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-common. 2024-04-26T04:29:20.405 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:20.421 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:20.445 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-osd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:20.461 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:20.515 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libradosstriper1. 2024-04-26T04:29:20.530 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:20.565 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libradosstriper1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:20.716 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-common. 2024-04-26T04:29:20.742 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:20.782 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:21.027 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-common. 2024-04-26T04:29:21.053 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:21.093 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:21.499 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph. 2024-04-26T04:29:21.526 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../066-ceph_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:21.557 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:21.862 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-fuse. 2024-04-26T04:29:21.886 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:21.935 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:21.950 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-base. 2024-04-26T04:29:21.977 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:22.026 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-base (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:22.154 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-base. 2024-04-26T04:29:22.181 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:22.246 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-base (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:22.313 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mds. 2024-04-26T04:29:22.339 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:22.379 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mds (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:22.565 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-base. 2024-04-26T04:29:22.579 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-26T04:29:22.592 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:22.606 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-26T04:29:22.641 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-base (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:22.646 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-26T04:29:22.757 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package cephadm. 2024-04-26T04:29:22.772 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../069-cephadm_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:22.807 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:22.833 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-26T04:29:22.860 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-26T04:29:22.899 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-26T04:29:22.949 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-cheroot. 2024-04-26T04:29:22.976 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-26T04:29:23.015 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-26T04:29:23.086 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-26T04:29:23.111 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-26T04:29:23.151 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-26T04:29:23.169 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-26T04:29:23.186 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-26T04:29:23.202 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cheroot. 2024-04-26T04:29:23.219 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-26T04:29:23.229 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-26T04:29:23.269 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-26T04:29:23.335 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-26T04:29:23.362 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-26T04:29:23.402 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-26T04:29:23.454 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-26T04:29:23.480 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-26T04:29:23.497 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-cheroot. 2024-04-26T04:29:23.521 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-26T04:29:23.523 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-26T04:29:23.563 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-26T04:29:23.572 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-26T04:29:23.599 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-26T04:29:23.638 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-26T04:29:23.680 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-26T04:29:23.706 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-26T04:29:23.746 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-26T04:29:23.884 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-26T04:29:23.899 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-26T04:29:23.910 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-26T04:29:23.926 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-26T04:29:23.950 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-26T04:29:23.957 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-26T04:29:24.049 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-26T04:29:24.076 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-26T04:29:24.116 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-26T04:29:24.244 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-26T04:29:24.252 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-26T04:29:24.271 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-26T04:29:24.279 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-26T04:29:24.310 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-26T04:29:24.319 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-26T04:29:24.418 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-tempora. 2024-04-26T04:29:24.445 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-26T04:29:24.485 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-26T04:29:24.501 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-babel. 2024-04-26T04:29:24.528 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-26T04:29:24.567 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-26T04:29:24.580 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-26T04:29:24.595 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-26T04:29:24.613 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-tempora. 2024-04-26T04:29:24.630 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-26T04:29:24.640 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-26T04:29:24.680 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-26T04:29:24.746 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-portend. 2024-04-26T04:29:24.773 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-26T04:29:24.804 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-26T04:29:24.879 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jinja2. 2024-04-26T04:29:24.905 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-26T04:29:24.908 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-tempora. 2024-04-26T04:29:24.934 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-26T04:29:24.945 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-26T04:29:24.974 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-26T04:29:24.983 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-portend. 2024-04-26T04:29:25.010 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-26T04:29:25.049 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-26T04:29:25.099 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-26T04:29:25.126 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-26T04:29:25.165 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-26T04:29:25.302 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-portend. 2024-04-26T04:29:25.319 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-26T04:29:25.330 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-26T04:29:25.346 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-26T04:29:25.369 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-26T04:29:25.385 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-26T04:29:25.452 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-26T04:29:25.478 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-26T04:29:25.482 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-26T04:29:25.509 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:25.518 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-26T04:29:25.548 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:25.672 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-26T04:29:25.688 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-26T04:29:25.699 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-26T04:29:25.715 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-26T04:29:25.738 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-26T04:29:25.755 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-26T04:29:25.855 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-natsort. 2024-04-26T04:29:25.882 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-26T04:29:25.894 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-26T04:29:25.920 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-26T04:29:25.921 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-26T04:29:25.960 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-26T04:29:26.000 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-26T04:29:26.026 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-26T04:29:26.058 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-26T04:29:26.125 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-natsort. 2024-04-26T04:29:26.152 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-26T04:29:26.191 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-26T04:29:26.208 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-logutils. 2024-04-26T04:29:26.223 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-26T04:29:26.238 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-routes. 2024-04-26T04:29:26.258 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-26T04:29:26.264 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-26T04:29:26.304 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-26T04:29:26.395 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-natsort. 2024-04-26T04:29:26.421 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-26T04:29:26.469 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-26T04:29:26.494 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-26T04:29:26.520 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-logutils. 2024-04-26T04:29:26.520 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-26T04:29:26.547 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-26T04:29:26.556 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-26T04:29:26.586 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-26T04:29:26.633 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-26T04:29:26.665 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:26.699 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:26.759 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-logutils. 2024-04-26T04:29:26.771 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-26T04:29:26.814 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-26T04:29:26.847 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-mako. 2024-04-26T04:29:26.847 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-26T04:29:26.873 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-26T04:29:26.874 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-26T04:29:26.913 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-26T04:29:26.914 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-26T04:29:27.084 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-26T04:29:27.112 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-26T04:29:27.150 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-26T04:29:27.183 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-mako. 2024-04-26T04:29:27.210 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-26T04:29:27.250 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-26T04:29:27.250 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-26T04:29:27.277 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-26T04:29:27.316 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-26T04:29:27.453 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-mako. 2024-04-26T04:29:27.479 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-26T04:29:27.519 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-26T04:29:27.578 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-26T04:29:27.602 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-26T04:29:27.605 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-26T04:29:27.629 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-26T04:29:27.645 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-26T04:29:27.669 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-26T04:29:27.848 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-26T04:29:27.875 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-26T04:29:27.914 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-26T04:29:27.947 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-26T04:29:27.955 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-26T04:29:27.974 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-26T04:29:27.983 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-26T04:29:27.988 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-webob. 2024-04-26T04:29:28.014 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-26T04:29:28.015 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-26T04:29:28.021 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-26T04:29:28.063 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-26T04:29:28.193 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-26T04:29:28.219 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-26T04:29:28.259 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-26T04:29:28.292 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webob. 2024-04-26T04:29:28.308 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-26T04:29:28.350 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-26T04:29:28.433 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-waitress. 2024-04-26T04:29:28.459 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-26T04:29:28.504 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-26T04:29:28.508 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-joblib. 2024-04-26T04:29:28.528 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-webob. 2024-04-26T04:29:28.535 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-26T04:29:28.554 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-26T04:29:28.574 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-26T04:29:28.595 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-26T04:29:28.653 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-waitress. 2024-04-26T04:29:28.680 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-26T04:29:28.715 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-26T04:29:28.786 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-tempita. 2024-04-26T04:29:28.812 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-26T04:29:28.844 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-26T04:29:28.920 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-26T04:29:28.931 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-waitress. 2024-04-26T04:29:28.947 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-26T04:29:28.958 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-26T04:29:28.986 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-26T04:29:29.009 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-26T04:29:29.023 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-tempita. 2024-04-26T04:29:29.050 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-26T04:29:29.106 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-26T04:29:29.123 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-paste. 2024-04-26T04:29:29.148 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-26T04:29:29.188 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-26T04:29:29.306 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-sklearn. 2024-04-26T04:29:29.309 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-tempita. 2024-04-26T04:29:29.333 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-26T04:29:29.335 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-26T04:29:29.372 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-26T04:29:29.375 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-26T04:29:29.409 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-paste. 2024-04-26T04:29:29.436 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-26T04:29:29.475 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-26T04:29:29.508 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-26T04:29:29.524 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-26T04:29:29.567 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-26T04:29:29.712 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-paste. 2024-04-26T04:29:29.738 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-26T04:29:29.778 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-26T04:29:29.829 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-26T04:29:29.844 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-26T04:29:29.856 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-26T04:29:29.859 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-26T04:29:29.894 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-26T04:29:29.912 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-26T04:29:30.067 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-26T04:29:30.094 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:30.107 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-26T04:29:30.123 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-26T04:29:30.165 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-26T04:29:30.173 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-webtest. 2024-04-26T04:29:30.181 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-26T04:29:30.198 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-26T04:29:30.199 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-26T04:29:30.216 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:30.234 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-26T04:29:30.240 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-26T04:29:30.467 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-26T04:29:30.494 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-26T04:29:30.533 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pecan. 2024-04-26T04:29:30.534 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webtest. 2024-04-26T04:29:30.534 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-26T04:29:30.560 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-26T04:29:30.561 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-26T04:29:30.600 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-26T04:29:30.601 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-26T04:29:30.787 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-webtest. 2024-04-26T04:29:30.802 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-26T04:29:30.837 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-26T04:29:30.879 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pecan. 2024-04-26T04:29:30.906 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-26T04:29:30.937 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-26T04:29:30.964 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-certifi. 2024-04-26T04:29:30.990 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-26T04:29:31.029 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-26T04:29:31.081 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pecan. 2024-04-26T04:29:31.108 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-26T04:29:31.156 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-26T04:29:31.170 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cachetools. 2024-04-26T04:29:31.198 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-26T04:29:31.237 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-26T04:29:31.317 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-certifi. 2024-04-26T04:29:31.344 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-26T04:29:31.358 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-idna. 2024-04-26T04:29:31.383 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-26T04:29:31.384 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-26T04:29:31.424 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-26T04:29:31.502 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-certifi. 2024-04-26T04:29:31.530 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-26T04:29:31.556 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-26T04:29:31.569 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-26T04:29:31.584 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-26T04:29:31.623 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-26T04:29:31.703 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-idna. 2024-04-26T04:29:31.719 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-urllib3. 2024-04-26T04:29:31.730 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-26T04:29:31.734 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-26T04:29:31.769 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-26T04:29:31.769 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-26T04:29:31.897 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-idna. 2024-04-26T04:29:31.924 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-26T04:29:31.935 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-26T04:29:31.963 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-26T04:29:31.963 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-26T04:29:32.002 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-26T04:29:32.064 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-urllib3. 2024-04-26T04:29:32.091 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-26T04:29:32.130 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-26T04:29:32.238 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-requests. 2024-04-26T04:29:32.263 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-rsa. 2024-04-26T04:29:32.264 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-26T04:29:32.283 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-urllib3. 2024-04-26T04:29:32.290 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-26T04:29:32.296 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-26T04:29:32.310 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-26T04:29:32.321 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-26T04:29:32.394 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-26T04:29:32.459 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-requests. 2024-04-26T04:29:32.486 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-26T04:29:32.525 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-26T04:29:32.550 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-26T04:29:32.576 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-26T04:29:32.616 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-26T04:29:32.624 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-google-auth. 2024-04-26T04:29:32.652 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-26T04:29:32.691 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-26T04:29:32.803 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-26T04:29:32.830 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-26T04:29:32.861 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-26T04:29:32.862 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-requests. 2024-04-26T04:29:32.891 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-26T04:29:32.894 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-26T04:29:32.921 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:32.928 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-26T04:29:32.952 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:33.011 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-26T04:29:33.038 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-26T04:29:33.078 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-26T04:29:33.156 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-26T04:29:33.184 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:33.222 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:33.239 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-26T04:29:33.265 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-26T04:29:33.305 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-26T04:29:33.326 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-26T04:29:33.352 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:33.389 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-websocket. 2024-04-26T04:29:33.392 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:33.416 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-26T04:29:33.456 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-26T04:29:33.583 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-26T04:29:33.586 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-26T04:29:33.599 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:33.613 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:33.642 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:33.652 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:33.662 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-26T04:29:33.688 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-26T04:29:33.729 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-26T04:29:33.751 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-26T04:29:33.778 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-26T04:29:33.818 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-26T04:29:33.989 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-26T04:29:34.013 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-26T04:29:34.015 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-26T04:29:34.029 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:34.031 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-openssl. 2024-04-26T04:29:34.055 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-26T04:29:34.058 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-26T04:29:34.071 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:34.098 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-26T04:29:34.358 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-openssl. 2024-04-26T04:29:34.385 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-26T04:29:34.399 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-26T04:29:34.424 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-26T04:29:34.426 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-26T04:29:34.435 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr. 2024-04-26T04:29:34.461 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:34.466 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-26T04:29:34.473 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-26T04:29:34.501 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:34.502 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:34.540 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:34.727 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr. 2024-04-26T04:29:34.754 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:34.769 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-openssl. 2024-04-26T04:29:34.785 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:34.795 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-26T04:29:34.835 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-26T04:29:34.846 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mon. 2024-04-26T04:29:34.851 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-26T04:29:34.873 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:34.879 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-26T04:29:34.904 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mon (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:34.917 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-26T04:29:35.121 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mgr. 2024-04-26T04:29:35.148 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:35.155 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mon. 2024-04-26T04:29:35.213 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mgr (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:35.214 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:35.223 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mon (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:35.237 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-26T04:29:35.265 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-26T04:29:35.338 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-26T04:29:35.509 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-osd. 2024-04-26T04:29:35.535 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:35.566 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mon. 2024-04-26T04:29:35.575 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-osd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:35.593 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:35.633 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mon (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:35.773 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package jq. 2024-04-26T04:29:35.801 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-26T04:29:35.840 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-26T04:29:35.867 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-osd. 2024-04-26T04:29:35.894 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:35.933 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-osd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:36.076 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package socat. 2024-04-26T04:29:36.104 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-26T04:29:36.151 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-26T04:29:36.236 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-osd. 2024-04-26T04:29:36.263 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:36.302 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-osd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:36.438 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package xmlstarlet. 2024-04-26T04:29:36.465 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-26T04:29:36.504 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-26T04:29:36.620 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph. 2024-04-26T04:29:36.636 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../066-ceph_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:36.670 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:36.975 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-test. 2024-04-26T04:29:37.003 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:37.041 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-test (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:37.103 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph. 2024-04-26T04:29:37.128 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../066-ceph_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:37.170 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:37.198 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-fuse. 2024-04-26T04:29:37.225 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:37.264 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:37.331 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph. 2024-04-26T04:29:37.358 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../066-ceph_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:37.391 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:37.498 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-fuse. 2024-04-26T04:29:37.525 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:37.589 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:37.635 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mds. 2024-04-26T04:29:37.662 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:37.701 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mds (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:37.709 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-fuse. 2024-04-26T04:29:37.725 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:37.767 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:37.967 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mds. 2024-04-26T04:29:37.995 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:38.034 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mds (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:38.063 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package cephadm. 2024-04-26T04:29:38.078 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../069-cephadm_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:38.112 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:38.162 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mds. 2024-04-26T04:29:38.188 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:38.228 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mds (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:38.395 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephadm. 2024-04-26T04:29:38.422 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../069-cephadm_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:38.449 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-26T04:29:38.461 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:38.476 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-26T04:29:38.515 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-26T04:29:38.606 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package cephadm. 2024-04-26T04:29:38.622 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../069-cephadm_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:38.655 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:38.764 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-26T04:29:38.792 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-26T04:29:38.830 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-26T04:29:38.843 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-26T04:29:38.871 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-26T04:29:38.935 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-26T04:29:38.977 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-26T04:29:39.004 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-26T04:29:39.044 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-26T04:29:39.125 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-26T04:29:39.141 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-26T04:29:39.183 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-26T04:29:39.346 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-volume. 2024-04-26T04:29:39.373 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:39.379 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-26T04:29:39.404 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-volume (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:39.407 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-26T04:29:39.446 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-26T04:29:39.691 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-26T04:29:39.707 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:39.742 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libcephfs-dev (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:40.007 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-babel. 2024-04-26T04:29:40.028 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-26T04:29:40.034 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-26T04:29:40.055 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-26T04:29:40.073 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-26T04:29:40.094 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-26T04:29:40.172 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-babel. 2024-04-26T04:29:40.199 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-26T04:29:40.239 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-26T04:29:40.386 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-jinja2. 2024-04-26T04:29:40.412 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-26T04:29:40.424 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-26T04:29:40.452 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-26T04:29:40.454 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-26T04:29:40.460 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-babel. 2024-04-26T04:29:40.487 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-26T04:29:40.490 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-26T04:29:40.526 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-26T04:29:40.557 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jinja2. 2024-04-26T04:29:40.585 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-26T04:29:40.624 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-26T04:29:40.796 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-26T04:29:40.823 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:40.846 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-jinja2. 2024-04-26T04:29:40.861 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package nvme-cli. 2024-04-26T04:29:40.863 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:40.872 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-26T04:29:40.888 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-26T04:29:40.912 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-26T04:29:40.927 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-26T04:29:40.977 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-26T04:29:41.004 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:41.044 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:41.200 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-26T04:29:41.206 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-26T04:29:41.227 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-26T04:29:41.233 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:41.265 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:41.266 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-26T04:29:41.325 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package pkg-config. 2024-04-26T04:29:41.352 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-26T04:29:41.380 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-26T04:29:41.391 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-26T04:29:41.407 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-26T04:29:41.455 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-26T04:29:41.545 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-routes. 2024-04-26T04:29:41.568 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-26T04:29:41.571 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-26T04:29:41.595 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-26T04:29:41.611 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-26T04:29:41.626 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-26T04:29:41.635 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-26T04:29:41.651 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-26T04:29:41.694 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-26T04:29:41.906 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-26T04:29:41.921 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-routes. 2024-04-26T04:29:41.934 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:41.948 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-26T04:29:41.964 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:41.987 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-26T04:29:41.991 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-routes. 2024-04-26T04:29:42.019 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-26T04:29:42.058 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-26T04:29:42.131 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-26T04:29:42.158 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-26T04:29:42.197 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-26T04:29:42.324 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-26T04:29:42.351 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:42.390 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:42.411 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-26T04:29:42.439 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:42.477 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:42.499 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pastescript. 2024-04-26T04:29:42.527 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-26T04:29:42.566 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-26T04:29:42.870 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pluggy. 2024-04-26T04:29:42.897 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-26T04:29:42.936 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-26T04:29:43.239 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-psutil. 2024-04-26T04:29:43.267 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-26T04:29:43.306 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-26T04:29:43.495 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-26T04:29:43.523 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-26T04:29:43.562 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-26T04:29:43.608 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-py. 2024-04-26T04:29:43.624 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-26T04:29:43.658 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-26T04:29:43.724 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-26T04:29:43.752 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-26T04:29:43.787 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-26T04:29:43.790 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-26T04:29:43.803 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-26T04:29:43.837 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-26T04:29:43.929 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pygments. 2024-04-26T04:29:43.956 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-26T04:29:43.987 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-26T04:29:44.040 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-joblib. 2024-04-26T04:29:44.067 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-26T04:29:44.098 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-26T04:29:44.286 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-joblib. 2024-04-26T04:29:44.307 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-joblib. 2024-04-26T04:29:44.314 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-26T04:29:44.334 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-26T04:29:44.352 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-26T04:29:44.373 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-26T04:29:44.408 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-26T04:29:44.424 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-26T04:29:44.457 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-26T04:29:44.460 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-26T04:29:44.488 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-26T04:29:44.527 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-26T04:29:44.714 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-26T04:29:44.718 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-26T04:29:44.734 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-26T04:29:44.742 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-26T04:29:44.760 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-toml. 2024-04-26T04:29:44.768 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-26T04:29:44.781 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-26T04:29:44.788 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-26T04:29:44.821 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-sklearn. 2024-04-26T04:29:44.827 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-26T04:29:44.849 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-26T04:29:44.888 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-26T04:29:45.013 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-sklearn. 2024-04-26T04:29:45.028 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-26T04:29:45.071 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-26T04:29:45.075 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn. 2024-04-26T04:29:45.103 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-26T04:29:45.130 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pytest. 2024-04-26T04:29:45.141 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-26T04:29:45.158 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-26T04:29:45.196 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-26T04:29:45.533 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-simplejson. 2024-04-26T04:29:45.561 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-26T04:29:45.599 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-26T04:29:45.630 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-26T04:29:45.658 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:45.791 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:45.821 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-26T04:29:45.837 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:45.877 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-26T04:29:45.883 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-26T04:29:45.893 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-26T04:29:45.912 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:45.936 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-26T04:29:45.956 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:46.036 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:46.673 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package radosgw. 2024-04-26T04:29:46.700 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../116-radosgw_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:46.739 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking radosgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:46.795 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-cachetools. 2024-04-26T04:29:46.823 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-26T04:29:46.852 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-cachetools. 2024-04-26T04:29:46.861 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-26T04:29:46.879 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-26T04:29:46.918 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-26T04:29:46.972 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cachetools. 2024-04-26T04:29:47.001 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-26T04:29:47.039 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-26T04:29:47.114 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-26T04:29:47.142 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-26T04:29:47.181 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-26T04:29:47.246 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-26T04:29:47.273 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-26T04:29:47.312 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-26T04:29:47.325 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-26T04:29:47.353 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-26T04:29:47.391 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-26T04:29:47.510 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-26T04:29:47.534 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package rbd-fuse. 2024-04-26T04:29:47.537 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-26T04:29:47.550 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:47.576 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-26T04:29:47.593 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking rbd-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:47.657 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-26T04:29:47.684 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-26T04:29:47.703 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-26T04:29:47.720 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-26T04:29:47.724 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-26T04:29:47.753 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-26T04:29:47.880 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-26T04:29:47.887 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-rsa. 2024-04-26T04:29:47.915 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-26T04:29:47.954 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-26T04:29:48.044 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-rsa. 2024-04-26T04:29:48.048 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rsa. 2024-04-26T04:29:48.071 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-26T04:29:48.075 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-26T04:29:48.110 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-26T04:29:48.114 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-26T04:29:48.131 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-26T04:29:48.257 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-26T04:29:48.275 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-google-auth. 2024-04-26T04:29:48.302 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-26T04:29:48.342 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-26T04:29:48.372 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-google-auth. 2024-04-26T04:29:48.388 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-26T04:29:48.422 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-26T04:29:48.434 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-google-auth. 2024-04-26T04:29:48.462 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-26T04:29:48.501 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-26T04:29:48.513 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-26T04:29:48.637 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-26T04:29:48.665 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-26T04:29:48.703 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-26T04:29:48.725 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-26T04:29:48.752 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-26T04:29:48.791 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-26T04:29:48.804 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-26T04:29:48.832 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-26T04:29:48.837 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-26T04:29:48.871 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-26T04:29:48.964 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-websocket. 2024-04-26T04:29:48.992 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-26T04:29:49.031 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-26T04:29:49.103 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-websocket. 2024-04-26T04:29:49.130 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-26T04:29:49.169 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-26T04:29:49.190 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-websocket. 2024-04-26T04:29:49.218 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-26T04:29:49.254 INFO:teuthology.orchestra.run.smithi113.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-26T04:29:49.257 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-26T04:29:49.319 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:49.360 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-26T04:29:49.387 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-26T04:29:49.451 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-26T04:29:49.481 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-26T04:29:49.508 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-26T04:29:49.547 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-26T04:29:49.577 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-26T04:29:49.605 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-26T04:29:49.643 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-26T04:29:49.827 INFO:teuthology.orchestra.run.smithi113.stdout:Adding system user cephadm....done 2024-04-26T04:29:49.967 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-26T04:29:50.107 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-26T04:29:50.136 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:50.173 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:50.212 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-26T04:29:50.235 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-26T04:29:50.239 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:50.278 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:50.349 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-26T04:29:50.366 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:50.407 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:50.493 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-26T04:29:50.494 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-26T04:29:50.521 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-26T04:29:50.539 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-26T04:29:50.560 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-26T04:29:50.566 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-26T04:29:50.597 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-26T04:29:50.621 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-26T04:29:50.710 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-26T04:29:50.738 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-26T04:29:50.785 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-26T04:29:50.871 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-26T04:29:50.887 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-26T04:29:50.897 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-26T04:29:50.921 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-26T04:29:50.942 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-26T04:29:50.969 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-26T04:29:51.008 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-26T04:29:51.071 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-26T04:29:51.096 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-26T04:29:51.138 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-26T04:29:51.140 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-26T04:29:51.166 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package jq. 2024-04-26T04:29:51.182 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-26T04:29:51.224 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-26T04:29:51.278 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package jq. 2024-04-26T04:29:51.305 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-26T04:29:51.352 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-26T04:29:51.382 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package jq. 2024-04-26T04:29:51.410 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-26T04:29:51.448 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-26T04:29:51.477 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package socat. 2024-04-26T04:29:51.493 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-26T04:29:51.535 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-26T04:29:51.558 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-26T04:29:51.597 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package socat. 2024-04-26T04:29:51.624 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-26T04:29:51.663 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-26T04:29:51.701 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package socat. 2024-04-26T04:29:51.729 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-26T04:29:51.767 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-26T04:29:51.848 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package xmlstarlet. 2024-04-26T04:29:51.851 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-26T04:29:51.874 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-26T04:29:51.913 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-26T04:29:51.950 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package xmlstarlet. 2024-04-26T04:29:51.966 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-26T04:29:51.999 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-26T04:29:52.073 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package xmlstarlet. 2024-04-26T04:29:52.090 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-26T04:29:52.131 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-26T04:29:52.211 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-26T04:29:52.285 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-test. 2024-04-26T04:29:52.301 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:52.343 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-test (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:52.354 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-test. 2024-04-26T04:29:52.370 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:52.403 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-test (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:52.462 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-26T04:29:52.493 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-test. 2024-04-26T04:29:52.510 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:52.552 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-test (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:52.730 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-26T04:29:52.965 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-26T04:29:53.199 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-26T04:29:53.434 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-26T04:29:53.685 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-ceph-argparse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:54.028 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-26T04:29:54.129 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-26T04:29:54.255 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-26T04:29:54.381 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-26T04:29:54.632 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-26T04:29:54.691 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package ceph-volume. 2024-04-26T04:29:54.719 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:54.749 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking ceph-volume (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:54.756 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package ceph-volume. 2024-04-26T04:29:54.783 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:54.822 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking ceph-volume (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:54.931 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-volume. 2024-04-26T04:29:54.959 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2457-gddeaa917-1jammy_all.deb ... 2024-04-26T04:29:54.997 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-volume (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:55.012 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-26T04:29:55.020 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-26T04:29:55.036 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:55.046 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-26T04:29:55.070 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking libcephfs-dev (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:55.110 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-26T04:29:55.127 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-26T04:29:55.154 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:55.193 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking libcephfs-dev (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:55.236 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-26T04:29:55.309 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-26T04:29:55.326 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:29:55.339 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-26T04:29:55.367 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-26T04:29:55.367 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs-dev (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:29:55.406 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-26T04:29:55.479 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-26T04:29:55.507 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-26T04:29:55.554 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-26T04:29:55.596 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-26T04:29:55.687 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-26T04:29:55.716 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-26T04:29:55.722 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-26T04:29:55.744 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-26T04:29:55.754 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-26T04:29:55.771 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-26T04:29:55.810 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-26T04:29:55.893 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-26T04:29:55.920 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-26T04:29:55.959 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-26T04:29:55.981 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-26T04:29:56.075 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-26T04:29:56.104 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-26T04:29:56.141 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-26T04:29:56.206 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package nvme-cli. 2024-04-26T04:29:56.233 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-26T04:29:56.241 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-26T04:29:56.272 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-26T04:29:56.330 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package nvme-cli. 2024-04-26T04:29:56.357 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-26T04:29:56.396 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-26T04:29:56.479 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package nvme-cli. 2024-04-26T04:29:56.507 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-26T04:29:56.545 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-26T04:29:56.643 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up zip (3.0-12build2) ... 2024-04-26T04:29:56.694 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package pkg-config. 2024-04-26T04:29:56.722 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-26T04:29:56.760 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-26T04:29:56.835 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package pkg-config. 2024-04-26T04:29:56.863 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-26T04:29:56.902 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-26T04:29:56.909 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-26T04:29:56.959 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package pkg-config. 2024-04-26T04:29:56.987 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-26T04:29:57.013 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-26T04:29:57.025 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-26T04:29:57.031 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-26T04:29:57.071 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-26T04:29:57.138 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-26T04:29:57.154 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-26T04:29:57.188 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-26T04:29:57.262 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-26T04:29:57.290 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-26T04:29:57.336 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-26T04:29:57.491 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-26T04:29:57.519 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-26T04:29:57.558 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-26T04:29:57.570 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-26T04:29:57.583 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-26T04:29:57.611 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-26T04:29:57.650 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-26T04:29:57.723 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-26T04:29:57.751 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-26T04:29:57.781 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-26T04:29:57.821 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-26T04:29:57.869 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pastescript. 2024-04-26T04:29:57.897 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-26T04:29:57.922 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-26T04:29:57.935 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-26T04:29:57.953 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pastescript. 2024-04-26T04:29:57.981 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-26T04:29:58.019 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-26T04:29:58.023 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-26T04:29:58.084 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastescript. 2024-04-26T04:29:58.112 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-26T04:29:58.150 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-26T04:29:58.238 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pluggy. 2024-04-26T04:29:58.266 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-26T04:29:58.305 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-26T04:29:58.339 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pluggy. 2024-04-26T04:29:58.354 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-26T04:29:58.367 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-26T04:29:58.406 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-26T04:29:58.491 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pluggy. 2024-04-26T04:29:58.515 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-26T04:29:58.550 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-psutil. 2024-04-26T04:29:58.553 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-26T04:29:58.577 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-26T04:29:58.616 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-26T04:29:58.726 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-psutil. 2024-04-26T04:29:58.742 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-26T04:29:58.753 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-26T04:29:58.792 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-26T04:29:58.848 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-psutil. 2024-04-26T04:29:58.876 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-26T04:29:58.914 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-26T04:29:58.978 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-py. 2024-04-26T04:29:59.002 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-26T04:29:59.005 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-26T04:29:59.044 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-26T04:29:59.145 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-py. 2024-04-26T04:29:59.173 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-26T04:29:59.220 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-26T04:29:59.251 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-py. 2024-04-26T04:29:59.279 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-26T04:29:59.314 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pygments. 2024-04-26T04:29:59.317 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-26T04:29:59.330 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-26T04:29:59.345 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-26T04:29:59.372 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-26T04:29:59.540 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pygments. 2024-04-26T04:29:59.568 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-26T04:29:59.604 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pygments. 2024-04-26T04:29:59.606 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-26T04:29:59.620 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-26T04:29:59.621 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-26T04:29:59.654 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-26T04:29:59.851 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-26T04:29:59.879 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-26T04:29:59.918 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-26T04:29:59.956 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-26T04:30:00.078 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-26T04:30:00.106 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-26T04:30:00.118 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-26T04:30:00.144 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-26T04:30:00.169 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-26T04:30:00.182 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-26T04:30:00.223 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-26T04:30:00.228 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-toml. 2024-04-26T04:30:00.244 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-26T04:30:00.278 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-26T04:30:00.350 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:00.464 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-toml. 2024-04-26T04:30:00.477 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-toml. 2024-04-26T04:30:00.492 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-26T04:30:00.505 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-26T04:30:00.530 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-26T04:30:00.531 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-pytest. 2024-04-26T04:30:00.543 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-26T04:30:00.546 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-26T04:30:00.581 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-26T04:30:00.634 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-idna (3.3-1) ... 2024-04-26T04:30:00.775 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-pytest. 2024-04-26T04:30:00.791 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-26T04:30:00.841 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-26T04:30:00.871 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pytest. 2024-04-26T04:30:00.899 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-26T04:30:00.926 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package python3-simplejson. 2024-04-26T04:30:00.937 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-26T04:30:00.952 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-26T04:30:00.953 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-26T04:30:00.992 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-26T04:30:01.212 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package python3-simplejson. 2024-04-26T04:30:01.240 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-26T04:30:01.261 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-26T04:30:01.277 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-26T04:30:01.278 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-26T04:30:01.283 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplejson. 2024-04-26T04:30:01.311 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-26T04:30:01.311 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-26T04:30:01.350 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-26T04:30:01.556 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-26T04:30:01.573 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-26T04:30:01.606 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-26T04:30:01.618 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-26T04:30:01.646 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-26T04:30:01.685 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-26T04:30:01.957 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package radosgw. 2024-04-26T04:30:01.985 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../116-radosgw_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:30:02.015 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking radosgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:02.124 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-26T04:30:02.353 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package radosgw. 2024-04-26T04:30:02.380 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../116-radosgw_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:30:02.418 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:30:02.419 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking radosgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:02.472 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package radosgw. 2024-04-26T04:30:02.501 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../116-radosgw_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:30:02.539 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking radosgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:02.569 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-26T04:30:02.826 INFO:teuthology.orchestra.run.smithi137.stdout:Selecting previously unselected package rbd-fuse. 2024-04-26T04:30:02.828 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-26T04:30:02.842 INFO:teuthology.orchestra.run.smithi137.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:30:02.885 INFO:teuthology.orchestra.run.smithi137.stdout:Unpacking rbd-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:02.955 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-26T04:30:03.081 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-26T04:30:03.188 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-26T04:30:03.222 INFO:teuthology.orchestra.run.smithi178.stdout:Selecting previously unselected package rbd-fuse. 2024-04-26T04:30:03.238 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:30:03.280 INFO:teuthology.orchestra.run.smithi178.stdout:Unpacking rbd-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:03.373 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-26T04:30:03.400 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package rbd-fuse. 2024-04-26T04:30:03.417 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2457-gddeaa917-1jammy_amd64.deb ... 2024-04-26T04:30:03.431 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-26T04:30:03.449 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking rbd-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:03.557 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-26T04:30:03.593 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-26T04:30:03.624 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-26T04:30:03.730 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-26T04:30:03.817 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-26T04:30:03.843 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-26T04:30:03.867 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:30:03.970 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-26T04:30:03.988 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-26T04:30:03.994 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-26T04:30:04.114 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-26T04:30:04.153 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-26T04:30:04.234 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-26T04:30:04.295 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-26T04:30:04.386 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-26T04:30:04.538 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-26T04:30:04.549 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-26T04:30:04.587 INFO:teuthology.orchestra.run.smithi137.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-26T04:30:04.753 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-26T04:30:04.797 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-26T04:30:04.901 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:04.981 INFO:teuthology.orchestra.run.smithi178.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-26T04:30:05.048 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-26T04:30:05.106 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:05.162 INFO:teuthology.orchestra.run.smithi012.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-26T04:30:05.266 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:05.299 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-26T04:30:05.392 INFO:teuthology.orchestra.run.smithi137.stdout:Adding system user cephadm....done 2024-04-26T04:30:05.523 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-26T04:30:05.567 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-26T04:30:05.668 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-26T04:30:05.833 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-26T04:30:05.895 INFO:teuthology.orchestra.run.smithi178.stdout:Adding system user cephadm....done 2024-04-26T04:30:05.925 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user cephadm....done 2024-04-26T04:30:05.928 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-26T04:30:06.054 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-26T04:30:06.068 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-26T04:30:06.111 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-26T04:30:06.156 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-26T04:30:06.218 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-26T04:30:06.321 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-26T04:30:06.423 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-26T04:30:06.445 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-26T04:30:06.623 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-26T04:30:06.666 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-26T04:30:06.696 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-26T04:30:06.767 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-26T04:30:06.792 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-26T04:30:06.805 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-26T04:30:06.874 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-26T04:30:07.000 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-26T04:30:07.025 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-26T04:30:07.043 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-26T04:30:07.090 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-26T04:30:07.117 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-26T04:30:07.135 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-26T04:30:07.190 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-26T04:30:07.295 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-26T04:30:07.386 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-26T04:30:07.419 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-26T04:30:07.421 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-26T04:30:07.450 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-26T04:30:07.545 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-26T04:30:07.638 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-26T04:30:07.706 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-26T04:30:07.710 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-26T04:30:07.763 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-26T04:30:07.930 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-26T04:30:07.987 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-26T04:30:08.065 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-26T04:30:08.067 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-26T04:30:08.190 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-26T04:30:08.205 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-26T04:30:08.318 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-26T04:30:08.326 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-26T04:30:08.450 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-26T04:30:08.455 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-26T04:30:08.575 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:30:08.594 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-26T04:30:08.684 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-26T04:30:08.693 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-26T04:30:08.732 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-26T04:30:08.946 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-26T04:30:08.961 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-26T04:30:08.991 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-ceph-argparse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:08.994 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-26T04:30:09.238 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-26T04:30:09.242 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-26T04:30:09.278 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-26T04:30:09.351 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-26T04:30:09.461 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-26T04:30:09.497 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-26T04:30:09.514 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-26T04:30:09.538 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-26T04:30:09.587 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-26T04:30:09.655 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-26T04:30:09.773 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-26T04:30:09.815 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-26T04:30:09.846 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-26T04:30:09.923 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-26T04:30:10.032 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-ceph-argparse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:10.074 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-argparse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:10.191 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-26T04:30:10.218 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-26T04:30:10.244 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-26T04:30:10.300 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-26T04:30:10.316 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-26T04:30:10.333 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-26T04:30:10.418 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-26T04:30:10.442 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-26T04:30:10.459 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-26T04:30:10.519 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-26T04:30:10.528 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-26T04:30:10.569 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-26T04:30:10.654 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-26T04:30:10.670 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-26T04:30:10.810 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-26T04:30:10.896 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-26T04:30:10.919 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-26T04:30:10.921 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-26T04:30:10.937 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-26T04:30:11.173 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-26T04:30:11.199 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-26T04:30:11.316 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-26T04:30:11.346 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-26T04:30:11.353 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-26T04:30:11.385 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-26T04:30:11.409 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-26T04:30:11.438 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-26T04:30:11.449 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-26T04:30:11.508 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-26T04:30:11.535 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-26T04:30:11.575 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-26T04:30:11.618 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up librados2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:11.719 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:11.730 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up zip (3.0-12build2) ... 2024-04-26T04:30:11.845 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-26T04:30:11.848 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-26T04:30:11.894 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-26T04:30:11.959 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-26T04:30:12.020 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-26T04:30:12.069 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-26T04:30:12.288 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-26T04:30:12.323 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libcephfs2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:12.362 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-26T04:30:12.499 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libradosstriper1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:12.564 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-26T04:30:12.575 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-26T04:30:12.625 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-26T04:30:12.696 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-26T04:30:12.853 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-26T04:30:12.866 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up zip (3.0-12build2) ... 2024-04-26T04:30:12.910 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up librbd1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:12.977 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-26T04:30:13.000 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-26T04:30:13.045 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-modules-core (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:13.128 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-26T04:30:13.171 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:13.330 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up zip (3.0-12build2) ... 2024-04-26T04:30:13.382 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-26T04:30:13.383 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-26T04:30:13.438 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-26T04:30:13.457 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-26T04:30:13.652 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-26T04:30:13.814 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-26T04:30:13.822 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libcephfs-dev (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:13.912 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-26T04:30:13.948 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-rados (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:14.013 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-26T04:30:14.056 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-26T04:30:14.074 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up librgw2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:14.131 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-26T04:30:14.158 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-26T04:30:14.200 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-26T04:30:14.416 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-26T04:30:14.418 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-26T04:30:14.440 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-26T04:30:14.552 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-26T04:30:14.678 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-26T04:30:14.701 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-26T04:30:14.852 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-26T04:30:14.988 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-26T04:30:15.044 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-26T04:30:15.095 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-26T04:30:15.303 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-26T04:30:15.380 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-26T04:30:15.387 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-rbd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:15.429 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:15.454 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-26T04:30:15.522 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up rbd-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:15.648 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-rgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:15.648 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-26T04:30:15.731 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-idna (3.3-1) ... 2024-04-26T04:30:15.747 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-26T04:30:15.774 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cephfs (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:15.900 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:16.016 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-26T04:30:16.040 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-26T04:30:16.090 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-26T04:30:16.318 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-26T04:30:16.341 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-26T04:30:16.393 INFO:teuthology.orchestra.run.smithi113.stdout:Adding group ceph....done 2024-04-26T04:30:16.459 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:16.669 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-26T04:30:16.761 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-idna (3.3-1) ... 2024-04-26T04:30:16.761 INFO:teuthology.orchestra.run.smithi113.stdout:Adding system user ceph....done 2024-04-26T04:30:16.930 INFO:teuthology.orchestra.run.smithi113.stdout:Setting system user ceph properties....done 2024-04-26T04:30:16.931 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-26T04:30:16.948 INFO:teuthology.orchestra.run.smithi113.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-26T04:30:17.054 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:17.078 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-26T04:30:17.081 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-26T04:30:17.254 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-26T04:30:17.352 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-26T04:30:17.364 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-idna (3.3-1) ... 2024-04-26T04:30:17.480 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:30:17.606 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-26T04:30:17.690 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-26T04:30:17.818 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-test (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:17.874 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-26T04:30:17.919 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up radosgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:17.975 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-26T04:30:18.076 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-26T04:30:18.217 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-26T04:30:18.361 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-26T04:30:18.377 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:30:18.377 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:30:18.460 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:30:18.578 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-26T04:30:18.612 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-26T04:30:18.821 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-26T04:30:18.839 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-base (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:18.846 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:30:18.879 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-26T04:30:18.947 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-26T04:30:18.976 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-26T04:30:19.073 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-26T04:30:19.130 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:30:19.175 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-26T04:30:19.265 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-26T04:30:19.273 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-26T04:30:19.524 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-26T04:30:19.533 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-26T04:30:19.583 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-26T04:30:19.599 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mds (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:19.633 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-26T04:30:19.759 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-26T04:30:19.784 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-26T04:30:19.831 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-26T04:30:19.832 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-26T04:30:19.842 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-26T04:30:20.035 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-26T04:30:20.077 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:30:20.086 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-26T04:30:20.194 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-26T04:30:20.251 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:20.295 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-26T04:30:20.354 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-26T04:30:20.456 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-26T04:30:20.456 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-26T04:30:20.487 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-26T04:30:20.537 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-26T04:30:20.605 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:30:20.664 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-26T04:30:20.731 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-26T04:30:20.746 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-26T04:30:20.919 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-osd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:20.956 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-26T04:30:21.006 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-26T04:30:21.057 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-26T04:30:21.082 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-26T04:30:21.189 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-26T04:30:21.189 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-26T04:30:21.265 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-26T04:30:21.308 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-26T04:30:21.383 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-26T04:30:21.525 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-26T04:30:21.568 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-26T04:30:21.604 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:21.676 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-26T04:30:21.729 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:21.784 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-26T04:30:21.818 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-26T04:30:21.881 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mon (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:21.894 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-26T04:30:21.944 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-26T04:30:22.070 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-26T04:30:22.070 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-26T04:30:22.093 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-26T04:30:22.094 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-26T04:30:22.161 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-26T04:30:22.196 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-26T04:30:22.262 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-26T04:30:22.284 INFO:teuthology.orchestra.run.smithi137.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-26T04:30:22.337 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-26T04:30:22.463 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-26T04:30:22.506 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-26T04:30:22.555 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-26T04:30:22.582 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:22.631 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-26T04:30:22.748 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-26T04:30:22.856 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-26T04:30:22.860 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:22.874 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-26T04:30:22.961 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-dashboard (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:23.016 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-26T04:30:23.096 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-volume (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:23.116 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-26T04:30:23.183 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-26T04:30:23.242 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-26T04:30:23.261 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-26T04:30:23.362 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-26T04:30:23.367 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-26T04:30:23.448 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-26T04:30:23.456 INFO:teuthology.orchestra.run.smithi178.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-26T04:30:23.478 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-26T04:30:23.566 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-26T04:30:23.613 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:30:23.703 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-26T04:30:23.727 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-26T04:30:23.739 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-26T04:30:23.755 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-26T04:30:23.812 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-26T04:30:23.854 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-26T04:30:23.979 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-26T04:30:24.057 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-26T04:30:24.068 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-26T04:30:24.188 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-26T04:30:24.322 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-26T04:30:24.358 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-26T04:30:24.448 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-26T04:30:24.448 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-26T04:30:24.574 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-26T04:30:24.617 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-26T04:30:24.743 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-26T04:30:24.825 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:30:24.833 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-26T04:30:24.878 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-26T04:30:24.942 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-26T04:30:25.019 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-26T04:30:25.101 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-26T04:30:25.210 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-26T04:30:25.277 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-26T04:30:25.321 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-26T04:30:25.461 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-26T04:30:25.553 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-26T04:30:25.571 INFO:teuthology.orchestra.run.smithi113.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-26T04:30:25.576 DEBUG:teuthology.parallel:result is None 2024-04-26T04:30:25.587 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-26T04:30:25.665 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-26T04:30:25.838 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-26T04:30:25.922 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-26T04:30:25.963 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-26T04:30:26.075 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-26T04:30:26.231 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-26T04:30:26.232 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-26T04:30:26.385 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-26T04:30:26.499 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-26T04:30:26.507 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-26T04:30:26.633 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-26T04:30:26.721 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-26T04:30:26.836 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-26T04:30:26.847 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up librados2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:26.909 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-26T04:30:26.973 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:27.099 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-26T04:30:27.185 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-26T04:30:27.221 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-26T04:30:27.369 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libcephfs2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:27.495 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libradosstriper1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:27.514 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-26T04:30:27.555 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-26T04:30:27.621 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-26T04:30:27.850 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-26T04:30:27.914 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up librbd1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:27.949 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-26T04:30:27.978 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up librados2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:28.040 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mgr-modules-core (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:28.102 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:28.166 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:28.211 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-26T04:30:28.250 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-26T04:30:28.369 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-26T04:30:28.369 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-26T04:30:28.514 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libcephfs2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:28.604 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-26T04:30:28.640 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libradosstriper1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:28.729 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librados2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:28.766 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-26T04:30:28.842 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up libcephfs-dev (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:28.855 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:28.943 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-rados (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:28.981 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-26T04:30:29.067 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up librbd1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:29.069 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up librgw2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:29.195 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-26T04:30:29.202 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-modules-core (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:29.317 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:29.320 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:29.426 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libradosstriper1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:29.522 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-26T04:30:29.522 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-26T04:30:29.544 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-26T04:30:29.820 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librbd1 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:29.947 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-modules-core (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:29.979 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up libcephfs-dev (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:30.073 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:30.105 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-rados (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:30.225 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up librgw2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:30.289 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-26T04:30:30.290 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-26T04:30:30.324 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-26T04:30:30.415 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-rbd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:30.575 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up rbd-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:30.701 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-rgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:30.732 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs-dev (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:30.827 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up python3-cephfs (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:30.858 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rados (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:30.945 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:30.992 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librgw2 (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:31.118 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-26T04:30:31.420 INFO:teuthology.orchestra.run.smithi137.stdout:Adding group ceph....done 2024-04-26T04:30:31.528 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-rbd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:31.654 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up rbd-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:31.779 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-rgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:31.897 INFO:teuthology.orchestra.run.smithi137.stdout:Adding system user ceph....done 2024-04-26T04:30:31.905 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up python3-cephfs (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:32.015 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:32.066 INFO:teuthology.orchestra.run.smithi137.stdout:Setting system user ceph properties....done 2024-04-26T04:30:32.082 INFO:teuthology.orchestra.run.smithi137.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-26T04:30:32.220 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-26T04:30:32.438 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rbd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:32.563 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-26T04:30:32.583 INFO:teuthology.orchestra.run.smithi178.stdout:Adding group ceph....done 2024-04-26T04:30:32.648 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up rbd-fuse (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:32.774 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:32.900 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cephfs (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:32.968 INFO:teuthology.orchestra.run.smithi178.stdout:Adding system user ceph....done 2024-04-26T04:30:33.010 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-common (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:33.136 INFO:teuthology.orchestra.run.smithi178.stdout:Setting system user ceph properties....done 2024-04-26T04:30:33.153 INFO:teuthology.orchestra.run.smithi178.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-26T04:30:33.272 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-test (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:33.290 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-26T04:30:33.406 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up radosgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:33.569 INFO:teuthology.orchestra.run.smithi012.stdout:Adding group ceph....done 2024-04-26T04:30:33.574 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-26T04:30:33.858 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:30:33.858 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:30:33.921 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user ceph....done 2024-04-26T04:30:34.032 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-test (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:34.098 INFO:teuthology.orchestra.run.smithi012.stdout:Setting system user ceph properties....done 2024-04-26T04:30:34.117 INFO:teuthology.orchestra.run.smithi012.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-26T04:30:34.142 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up radosgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:34.260 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-26T04:30:34.347 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-base (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:34.587 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:30:34.587 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:30:34.616 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-26T04:30:34.902 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-26T04:30:35.044 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-base (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:35.077 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-test (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:35.237 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up radosgw (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:35.398 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-26T04:30:35.700 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mds (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:35.723 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:30:35.723 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:30:35.870 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mds (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:35.985 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-26T04:30:35.985 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-26T04:30:36.116 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-26T04:30:36.116 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-26T04:30:36.147 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-base (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:36.430 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mgr (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:36.509 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-26T04:30:36.629 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-26T04:30:36.629 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-26T04:30:36.630 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:36.837 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-26T04:30:36.894 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-26T04:30:36.965 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mds (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:37.089 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-osd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:37.197 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-26T04:30:37.197 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-26T04:30:37.299 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-osd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:37.373 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-26T04:30:37.374 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-26T04:30:37.566 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-26T04:30:37.566 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-26T04:30:37.642 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:37.849 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:37.854 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-26T04:30:37.854 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-26T04:30:37.975 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:38.075 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:38.118 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mon (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:38.210 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:38.310 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-osd (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:38.322 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-26T04:30:38.322 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-26T04:30:38.361 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mon (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:38.576 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-26T04:30:38.576 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-26T04:30:38.602 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-26T04:30:38.602 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-26T04:30:38.778 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mgr-cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:38.895 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:39.021 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-mgr-dashboard (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:39.037 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:39.071 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:39.188 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:39.196 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:39.314 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-mgr-dashboard (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:39.347 INFO:teuthology.orchestra.run.smithi137.stdout:Setting up ceph-volume (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:39.348 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mon (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:39.465 INFO:teuthology.orchestra.run.smithi178.stdout:Setting up ceph-volume (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:39.567 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-26T04:30:39.567 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-26T04:30:39.700 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-26T04:30:39.801 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-26T04:30:39.809 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-26T04:30:39.945 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-26T04:30:40.039 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-26T04:30:40.065 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-cephadm (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:40.199 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-26T04:30:40.217 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:40.343 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-dashboard (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:40.493 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-volume (19.0.0-2457-gddeaa917-1jammy) ... 2024-04-26T04:30:40.720 INFO:teuthology.orchestra.run.smithi137.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-26T04:30:40.863 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-26T04:30:40.955 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-26T04:30:41.108 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-26T04:30:41.216 INFO:teuthology.orchestra.run.smithi178.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-26T04:30:41.400 INFO:teuthology.orchestra.run.smithi137.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-26T04:30:41.405 DEBUG:teuthology.parallel:result is None 2024-04-26T04:30:41.878 INFO:teuthology.orchestra.run.smithi178.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-26T04:30:41.882 DEBUG:teuthology.parallel:result is None 2024-04-26T04:30:42.208 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-26T04:30:42.863 INFO:teuthology.orchestra.run.smithi012.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-26T04:30:42.868 DEBUG:teuthology.parallel:result is None 2024-04-26T04:30:42.868 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T04:30:43.011 DEBUG:teuthology.orchestra.run.smithi012:> dpkg-query -W -f '${Version}' ceph 2024-04-26T04:30:43.030 INFO:teuthology.orchestra.run.smithi012.stdout:19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:30:43.031 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:30:43.031 INFO:teuthology.task.install:The correct ceph version 19.0.0-2457-gddeaa917-1jammy is installed. 2024-04-26T04:30:43.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T04:30:43.266 DEBUG:teuthology.orchestra.run.smithi113:> dpkg-query -W -f '${Version}' ceph 2024-04-26T04:30:43.285 INFO:teuthology.orchestra.run.smithi113.stdout:19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:30:43.285 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:30:43.285 INFO:teuthology.task.install:The correct ceph version 19.0.0-2457-gddeaa917-1jammy is installed. 2024-04-26T04:30:43.287 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T04:30:43.529 DEBUG:teuthology.orchestra.run.smithi137:> dpkg-query -W -f '${Version}' ceph 2024-04-26T04:30:43.548 INFO:teuthology.orchestra.run.smithi137.stdout:19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:30:43.548 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:30:43.548 INFO:teuthology.task.install:The correct ceph version 19.0.0-2457-gddeaa917-1jammy is installed. 2024-04-26T04:30:43.550 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ddeaa91797339a19172c3036ff48cca58c12f448 2024-04-26T04:30:43.795 DEBUG:teuthology.orchestra.run.smithi178:> dpkg-query -W -f '${Version}' ceph 2024-04-26T04:30:43.814 INFO:teuthology.orchestra.run.smithi178.stdout:19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:30:43.814 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2457-gddeaa917-1jammy 2024-04-26T04:30:43.814 INFO:teuthology.task.install:The correct ceph version 19.0.0-2457-gddeaa917-1jammy is installed. 2024-04-26T04:30:43.816 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T04:30:43.816 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:30:43.817 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T04:30:43.831 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:30:43.832 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T04:30:43.846 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:30:43.846 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T04:30:43.860 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:30:43.861 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T04:30:43.875 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T04:30:43.875 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:30:43.875 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T04:30:43.891 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T04:30:43.946 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:30:43.947 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T04:30:43.960 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T04:30:44.015 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:30:44.015 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T04:30:44.028 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T04:30:44.082 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:30:44.082 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T04:30:44.096 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T04:30:44.150 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T04:30:44.150 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:30:44.151 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T04:30:44.164 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T04:30:44.219 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:30:44.219 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T04:30:44.233 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T04:30:44.286 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:30:44.286 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T04:30:44.300 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T04:30:44.355 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:30:44.355 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T04:30:44.404 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T04:30:44.458 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T04:30:44.458 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:30:44.458 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T04:30:44.472 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T04:30:44.527 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:30:44.527 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T04:30:44.541 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T04:30:44.595 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:30:44.595 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T04:30:44.608 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T04:30:44.663 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:30:44.663 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T04:30:44.676 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T04:30:44.731 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T04:30:44.825 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T04:30:44.825 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /var/log/ceph 2024-04-26T04:30:44.828 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /var/log/ceph 2024-04-26T04:30:44.830 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /var/log/ceph 2024-04-26T04:30:44.832 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /var/log/ceph 2024-04-26T04:30:44.845 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T04:30:44.846 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T04:30:44.884 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T04:30:44.887 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T04:30:44.889 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T04:30:44.901 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T04:30:44.901 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T04:30:44.948 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T04:30:44.950 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T04:30:44.953 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T04:30:44.981 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T04:30:44.982 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon election default strategy': 3, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T04:30:44.982 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537', 'branch': 'wip-yuri2-testing-2024-04-24-0914-squid', 'description': 'rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7674537', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri2-testing-2024-04-24-0914-squid'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri2-testing-2024-04-24-0914-squid', 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'], ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'], ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'], ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0']], 'seed': 1848, 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri2-testing-2024-04-24-0914-squid', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_ddeaa91797339a19172c3036ff48cca58c12f448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448', 'targets': {'smithi012.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOpPyOBmkp02ua0j8uAGE6mVom8u8MxXYf8hD9EdS8KvAiHHjzMROwJtse/ELhFk77iWt7NO4IUcqGpl0EyUBQ0=', 'smithi113.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDtaKlZyDmKaDMPdgJlVIfpH2fxSXBxeeltzobA7Qh+t5mQBLUto9ttQ8myJEXBOqOZN/3rCBHUaAv3Gjf5n7qA=', 'smithi137.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNY+ds4buWbZuY9lv3VjkNF/ay4yilCJKIAfwijdXptA/UX6nyKxz05LNhuN11GOW2ZzgYckEYKvXqECqcH23Vs=', 'smithi178.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLwLyDbrNL+/xuKJZRbJCPASorcyTzEoDK8CiexqrA/fptcNO/IYz2WZAnTIz2mNwNMJhTvehJaC2sJpcNJJSgw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon election default strategy': 3, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448', 'cluster': 'ceph'}}, {'thrashosds': {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200}}, {'rados': {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_01:23:56', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T04:30:44.982 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T04:30:45.000 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T04:30:45.012 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T04:30:45.015 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T04:30:45.020 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T04:30:45.048 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T04:30:45.060 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T04:30:45.064 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T04:30:45.079 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:30:45.079 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T04:30:45.107 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T04:30:45.107 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-04-26T04:30:45.156 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T04:30:45.157 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:45.157 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 885 Links: 1 2024-04-26T04:30:45.157 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:45.157 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-26 04:24:53.219812890 +0000 2024-04-26T04:30:45.157 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-26 04:24:52.967815142 +0000 2024-04-26T04:30:45.157 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-26 04:24:52.967815142 +0000 2024-04-26T04:30:45.157 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-04-26T04:30:45.157 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T04:30:45.211 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-26T04:30:45.211 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-26T04:30:45.211 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000352066 s, 1.5 MB/s 2024-04-26T04:30:45.213 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T04:30:45.262 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-04-26T04:30:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T04:30:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:45.307 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 903 Links: 1 2024-04-26T04:30:45.307 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:45.307 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-26 04:24:53.739808246 +0000 2024-04-26T04:30:45.307 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-26 04:24:53.479810568 +0000 2024-04-26T04:30:45.307 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-26 04:24:53.479810568 +0000 2024-04-26T04:30:45.307 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-04-26T04:30:45.308 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T04:30:45.362 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-26T04:30:45.362 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-26T04:30:45.362 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000333881 s, 1.5 MB/s 2024-04-26T04:30:45.363 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T04:30:45.412 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-04-26T04:30:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T04:30:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:45.460 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-04-26T04:30:45.460 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:45.460 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-26 04:24:54.231803850 +0000 2024-04-26T04:30:45.460 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-26 04:24:53.975806138 +0000 2024-04-26T04:30:45.460 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-26 04:24:53.975806138 +0000 2024-04-26T04:30:45.460 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-04-26T04:30:45.461 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T04:30:45.514 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-26T04:30:45.514 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-26T04:30:45.514 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000402074 s, 1.3 MB/s 2024-04-26T04:30:45.516 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T04:30:45.565 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-04-26T04:30:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T04:30:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:45.611 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 5h/5d Inode: 934 Links: 1 2024-04-26T04:30:45.611 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:45.612 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-04-26 04:24:54.495801493 +0000 2024-04-26T04:30:45.612 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-04-26 04:24:54.491801528 +0000 2024-04-26T04:30:45.612 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-04-26 04:24:54.491801528 +0000 2024-04-26T04:30:45.612 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-04-26T04:30:45.612 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T04:30:45.666 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-04-26T04:30:45.666 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-04-26T04:30:45.666 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.00033163 s, 1.5 MB/s 2024-04-26T04:30:45.668 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T04:30:45.716 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-26T04:30:45.717 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:30:45.717 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T04:30:45.722 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T04:30:45.722 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2024-04-26T04:30:45.772 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T04:30:45.772 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:45.772 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-04-26T04:30:45.772 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:45.772 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-26 04:24:53.907983741 +0000 2024-04-26T04:30:45.772 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-26 04:24:53.659985954 +0000 2024-04-26T04:30:45.772 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-26 04:24:53.659985954 +0000 2024-04-26T04:30:45.772 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: - 2024-04-26T04:30:45.773 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T04:30:45.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-26T04:30:45.827 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-26T04:30:45.827 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000438951 s, 1.2 MB/s 2024-04-26T04:30:45.829 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T04:30:45.877 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2024-04-26T04:30:45.924 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T04:30:45.924 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:45.924 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 899 Links: 1 2024-04-26T04:30:45.924 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:45.924 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-26 04:24:54.559977924 +0000 2024-04-26T04:30:45.924 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-26 04:24:54.135981707 +0000 2024-04-26T04:30:45.924 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-26 04:24:54.135981707 +0000 2024-04-26T04:30:45.924 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: - 2024-04-26T04:30:45.924 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T04:30:45.978 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-26T04:30:45.978 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-26T04:30:45.978 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000315404 s, 1.6 MB/s 2024-04-26T04:30:45.979 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T04:30:46.033 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2024-04-26T04:30:46.079 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T04:30:46.079 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:46.079 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 922 Links: 1 2024-04-26T04:30:46.079 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:46.079 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-26 04:24:54.787975889 +0000 2024-04-26T04:30:46.080 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-26 04:24:54.783975926 +0000 2024-04-26T04:30:46.080 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-26 04:24:54.783975926 +0000 2024-04-26T04:30:46.080 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: - 2024-04-26T04:30:46.080 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T04:30:46.133 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-26T04:30:46.133 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-26T04:30:46.134 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000429423 s, 1.2 MB/s 2024-04-26T04:30:46.135 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T04:30:46.185 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2024-04-26T04:30:46.231 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T04:30:46.231 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:46.231 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 939 Links: 1 2024-04-26T04:30:46.231 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:46.232 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-04-26 04:24:55.535969218 +0000 2024-04-26T04:30:46.232 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-04-26 04:24:55.259971680 +0000 2024-04-26T04:30:46.232 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-04-26 04:24:55.259971680 +0000 2024-04-26T04:30:46.232 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: - 2024-04-26T04:30:46.232 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T04:30:46.285 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-04-26T04:30:46.285 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-04-26T04:30:46.286 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.00038491 s, 1.3 MB/s 2024-04-26T04:30:46.287 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T04:30:46.337 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-26T04:30:46.337 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:30:46.337 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T04:30:46.343 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T04:30:46.344 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-04-26T04:30:46.391 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T04:30:46.392 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:46.392 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 890 Links: 1 2024-04-26T04:30:46.392 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:46.392 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-26 04:24:55.128465496 +0000 2024-04-26T04:30:46.392 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-26 04:24:54.876467747 +0000 2024-04-26T04:30:46.392 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-26 04:24:54.876467747 +0000 2024-04-26T04:30:46.392 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-04-26T04:30:46.392 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T04:30:46.446 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-26T04:30:46.446 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-26T04:30:46.446 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000433016 s, 1.2 MB/s 2024-04-26T04:30:46.447 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T04:30:46.496 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-04-26T04:30:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T04:30:46.544 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:46.545 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 906 Links: 1 2024-04-26T04:30:46.545 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:46.545 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-26 04:24:55.344463567 +0000 2024-04-26T04:30:46.545 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-26 04:24:55.340463603 +0000 2024-04-26T04:30:46.545 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-26 04:24:55.340463603 +0000 2024-04-26T04:30:46.545 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-04-26T04:30:46.545 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T04:30:46.597 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-26T04:30:46.598 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-26T04:30:46.598 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000368169 s, 1.4 MB/s 2024-04-26T04:30:46.599 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T04:30:46.648 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-04-26T04:30:46.696 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T04:30:46.696 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:46.696 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 923 Links: 1 2024-04-26T04:30:46.696 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:46.696 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-26 04:24:56.124456599 +0000 2024-04-26T04:30:46.697 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-26 04:24:55.864458921 +0000 2024-04-26T04:30:46.697 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-26 04:24:55.864458921 +0000 2024-04-26T04:30:46.697 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-04-26T04:30:46.698 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T04:30:46.749 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-26T04:30:46.749 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-26T04:30:46.749 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000366473 s, 1.4 MB/s 2024-04-26T04:30:46.750 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T04:30:46.800 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-04-26T04:30:46.847 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T04:30:46.848 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:46.848 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 939 Links: 1 2024-04-26T04:30:46.848 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:46.848 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-26 04:24:56.372454383 +0000 2024-04-26T04:30:46.848 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-26 04:24:56.368454420 +0000 2024-04-26T04:30:46.848 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-26 04:24:56.368454420 +0000 2024-04-26T04:30:46.848 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: - 2024-04-26T04:30:46.848 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T04:30:46.902 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-26T04:30:46.902 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-26T04:30:46.902 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000315912 s, 1.6 MB/s 2024-04-26T04:30:46.903 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T04:30:46.952 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-26T04:30:46.952 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:30:46.953 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T04:30:46.959 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T04:30:46.959 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-04-26T04:30:47.007 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T04:30:47.007 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:47.008 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 882 Links: 1 2024-04-26T04:30:47.008 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:47.008 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-26 04:24:57.177718250 +0000 2024-04-26T04:30:47.008 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-26 04:24:56.897720751 +0000 2024-04-26T04:30:47.008 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-26 04:24:56.897720751 +0000 2024-04-26T04:30:47.008 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2024-04-26T04:30:47.008 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T04:30:47.062 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-26T04:30:47.062 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-26T04:30:47.062 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000317034 s, 1.6 MB/s 2024-04-26T04:30:47.063 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T04:30:47.115 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-04-26T04:30:47.164 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T04:30:47.164 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:47.164 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-04-26T04:30:47.165 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:47.165 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-26 04:24:57.437715928 +0000 2024-04-26T04:30:47.165 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-26 04:24:57.433715964 +0000 2024-04-26T04:30:47.165 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-26 04:24:57.433715964 +0000 2024-04-26T04:30:47.165 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2024-04-26T04:30:47.166 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T04:30:47.226 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-26T04:30:47.226 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-26T04:30:47.226 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000308651 s, 1.7 MB/s 2024-04-26T04:30:47.227 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T04:30:47.276 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-04-26T04:30:47.323 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T04:30:47.323 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:47.323 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-04-26T04:30:47.323 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:47.323 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-26 04:24:58.197709139 +0000 2024-04-26T04:30:47.323 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-26 04:24:57.937711461 +0000 2024-04-26T04:30:47.323 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-26 04:24:57.937711461 +0000 2024-04-26T04:30:47.323 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2024-04-26T04:30:47.323 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T04:30:47.377 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-26T04:30:47.377 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-26T04:30:47.377 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000301727 s, 1.7 MB/s 2024-04-26T04:30:47.378 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T04:30:47.428 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-04-26T04:30:47.475 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T04:30:47.475 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:30:47.475 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 933 Links: 1 2024-04-26T04:30:47.475 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:30:47.475 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-04-26 04:24:58.685704781 +0000 2024-04-26T04:30:47.475 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-04-26 04:24:58.429707067 +0000 2024-04-26T04:30:47.475 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-04-26 04:24:58.429707067 +0000 2024-04-26T04:30:47.475 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2024-04-26T04:30:47.475 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T04:30:47.529 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-04-26T04:30:47.529 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-04-26T04:30:47.529 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000302525 s, 1.7 MB/s 2024-04-26T04:30:47.530 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T04:30:47.580 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-26T04:30:47.580 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi137.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-04-26T04:30:47.580 INFO:tasks.ceph:Generating config... 2024-04-26T04:30:47.582 INFO:tasks.ceph:[osd] debug monc = 20 2024-04-26T04:30:47.582 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-26T04:30:47.582 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-26T04:30:47.582 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T04:30:47.582 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T04:30:47.582 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-04-26T04:30:47.582 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-26T04:30:47.582 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-26T04:30:47.582 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T04:30:47.582 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T04:30:47.582 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-26T04:30:47.583 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T04:30:47.583 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T04:30:47.583 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T04:30:47.583 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-26T04:30:47.583 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.1 2024-04-26T04:30:47.583 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-04-26T04:30:47.583 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-04-26T04:30:47.583 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T04:30:47.583 INFO:tasks.ceph:[osd] osd max backfills = 3 2024-04-26T04:30:47.583 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-26T04:30:47.583 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-04-26T04:30:47.583 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-04-26T04:30:47.583 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-04-26T04:30:47.584 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T04:30:47.584 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-04-26T04:30:47.584 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-04-26T04:30:47.584 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-04-26T04:30:47.584 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-04-26T04:30:47.584 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-26T04:30:47.584 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-04-26T04:30:47.584 INFO:tasks.ceph:[global] osd debug inject dispatch delay duration = 0.1 2024-04-26T04:30:47.584 INFO:tasks.ceph:[global] osd debug inject dispatch delay probability = 0.1 2024-04-26T04:30:47.584 INFO:tasks.ceph:[global] osd_async_recovery_min_cost = 1 2024-04-26T04:30:47.584 INFO:tasks.ceph:[global] osd_object_clean_region_max_num_intervals = 1000 2024-04-26T04:30:47.584 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T04:30:47.584 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T04:30:47.584 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T04:30:47.585 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T04:30:47.585 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T04:30:47.585 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-04-26T04:30:47.585 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T04:30:47.585 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T04:30:47.622 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T04:30:47.638 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T04:30:47.714 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T04:30:47.771 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.12'), ('mon.b', '172.21.15.113'), ('mon.c', '172.21.15.137')] 2024-04-26T04:30:47.771 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.12,172.21.15.113,172.21.15.137', 'mon election default strategy': 3, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-04-26T04:30:47.772 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:30:47.772 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T04:30:47.819 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.12 --add b 172.21.15.113 --add c 172.21.15.137 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:30:47.907 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:30:47.907 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: generated fsid 12c27fbe-0161-48d0-88ec-42a226685437 2024-04-26T04:30:47.907 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = quincy 2024-04-26T04:30:47.907 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 0 2024-04-26T04:30:47.907 INFO:teuthology.orchestra.run.smithi012.stdout:fsid 12c27fbe-0161-48d0-88ec-42a226685437 2024-04-26T04:30:47.907 INFO:teuthology.orchestra.run.smithi012.stdout:last_changed 2024-04-26T04:30:47.905217+0000 2024-04-26T04:30:47.907 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-04-26T04:30:47.905217+0000 2024-04-26T04:30:47.907 INFO:teuthology.orchestra.run.smithi012.stdout:min_mon_release 17 (quincy) 2024-04-26T04:30:47.907 INFO:teuthology.orchestra.run.smithi012.stdout:election_strategy: 3 2024-04-26T04:30:47.907 INFO:teuthology.orchestra.run.smithi012.stdout:0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-04-26T04:30:47.908 INFO:teuthology.orchestra.run.smithi012.stdout:1: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.b 2024-04-26T04:30:47.908 INFO:teuthology.orchestra.run.smithi012.stdout:2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.c 2024-04-26T04:30:47.908 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T04:30:47.909 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T04:30:47.914 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 12c27fbe-0161-48d0-88ec-42a226685437... 2024-04-26T04:30:47.915 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T04:30:47.960 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T04:30:47.962 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T04:30:47.965 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T04:30:47.986 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-04-26T04:30:47.986 INFO:teuthology.orchestra.run.smithi012.stdout: chdir = "" 2024-04-26T04:30:47.986 INFO:teuthology.orchestra.run.smithi012.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T04:30:47.986 INFO:teuthology.orchestra.run.smithi012.stdout: auth supported = cephx 2024-04-26T04:30:47.986 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.986 INFO:teuthology.orchestra.run.smithi012.stdout: filestore xattr use omap = true 2024-04-26T04:30:47.986 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.986 INFO:teuthology.orchestra.run.smithi012.stdout: mon clock drift allowed = 1.000 2024-04-26T04:30:47.986 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.987 INFO:teuthology.orchestra.run.smithi012.stdout: osd crush chooseleaf type = 0 2024-04-26T04:30:47.987 INFO:teuthology.orchestra.run.smithi012.stdout: auth debug = true 2024-04-26T04:30:47.987 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.987 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on old message = true 2024-04-26T04:30:47.987 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on bug = true 2024-04-26T04:30:47.987 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.987 INFO:teuthology.orchestra.run.smithi012.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T04:30:47.987 INFO:teuthology.orchestra.run.smithi012.stdout: mon pg warn max object skew = 0 2024-04-26T04:30:47.987 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.987 INFO:teuthology.orchestra.run.smithi012.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T04:30:47.987 INFO:teuthology.orchestra.run.smithi012.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T04:30:47.987 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.987 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default size = 2 2024-04-26T04:30:47.988 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.988 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow primary affinity = true 2024-04-26T04:30:47.988 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow pg remap = true 2024-04-26T04:30:47.988 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on legacy crush tunables = false 2024-04-26T04:30:47.988 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on crush straw calc version zero = false 2024-04-26T04:30:47.988 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on no sortbitwise = false 2024-04-26T04:30:47.988 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on osd down out interval zero = false 2024-04-26T04:30:47.988 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on too few osds = false 2024-04-26T04:30:47.988 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T04:30:47.988 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T04:30:47.988 INFO:teuthology.orchestra.run.smithi012.stdout: mon_allow_pool_size_one = true 2024-04-26T04:30:47.988 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.988 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: osd default data pool replay window = 5 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: mon allow pool delete = true 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: mon cluster log file level = debug 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: debug asserts on shutdown = true 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: mon health detail to clog = false 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: mon host = "172.21.15.12,172.21.15.113,172.21.15.137" 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: mon election default strategy = 3 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug inject dispatch delay duration = 0.1 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug inject dispatch delay probability = 0.1 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: osd_async_recovery_min_cost = 1 2024-04-26T04:30:47.989 INFO:teuthology.orchestra.run.smithi012.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-04-26T04:30:47.990 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = 12c27fbe-0161-48d0-88ec-42a226685437 2024-04-26T04:30:47.990 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.990 INFO:teuthology.orchestra.run.smithi012.stdout:[osd] 2024-04-26T04:30:47.990 INFO:teuthology.orchestra.run.smithi012.stdout: osd journal size = 100 2024-04-26T04:30:47.990 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.990 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub load threshold = 5.0 2024-04-26T04:30:47.990 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub max interval = 120 2024-04-26T04:30:47.990 INFO:teuthology.orchestra.run.smithi012.stdout: osd mclock profile = high_recovery_ops 2024-04-26T04:30:47.990 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.990 INFO:teuthology.orchestra.run.smithi012.stdout: osd recover clone overlap = true 2024-04-26T04:30:47.990 INFO:teuthology.orchestra.run.smithi012.stdout: osd recovery max chunk = 1048576 2024-04-26T04:30:47.990 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.990 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug shutdown = true 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug op order = true 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug verify stray on activate = true 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug trim objects = true 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: osd open classes on start = true 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug pg log writeout = true 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: osd deep scrub update digest min age = 30 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: osd map max advance = 10 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: journal zero on create = true 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.991 INFO:teuthology.orchestra.run.smithi012.stdout: filestore ondisk finisher threads = 3 2024-04-26T04:30:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: filestore apply finisher threads = 3 2024-04-26T04:30:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: bdev debug aio = true 2024-04-26T04:30:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug misdirected ops = true 2024-04-26T04:30:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: debug monc = 20 2024-04-26T04:30:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: bdev async discard = True 2024-04-26T04:30:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: bdev enable discard = True 2024-04-26T04:30:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore block size = 96636764160 2024-04-26T04:30:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore fsck on mount = True 2024-04-26T04:30:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore zero block detection = True 2024-04-26T04:30:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluefs = 20 2024-04-26T04:30:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluestore = 20 2024-04-26T04:30:47.992 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: debug osd = 20 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: debug rocksdb = 10 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd full ratio = 0.9 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd nearfull ratio = 0.8 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: osd blocked scrub grace period = 3600 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug reject backfill probability = 0.1 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug verify cached snaps = True 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug verify missing on start = True 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: osd failsafe full ratio = 0.95 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: osd max backfills = 3 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: osd max markdown count = 1000 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: osd mclock override recovery settings = True 2024-04-26T04:30:47.993 INFO:teuthology.orchestra.run.smithi012.stdout: osd memory target = 1610612736 2024-04-26T04:30:47.994 INFO:teuthology.orchestra.run.smithi012.stdout: osd objectstore = bluestore 2024-04-26T04:30:47.994 INFO:teuthology.orchestra.run.smithi012.stdout: osd op queue = debug_random 2024-04-26T04:30:47.994 INFO:teuthology.orchestra.run.smithi012.stdout: osd op queue cut off = debug_random 2024-04-26T04:30:47.994 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub min interval = 60 2024-04-26T04:30:47.994 INFO:teuthology.orchestra.run.smithi012.stdout: osd shutdown pgref assert = True 2024-04-26T04:30:47.994 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.994 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr] 2024-04-26T04:30:47.994 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-04-26T04:30:47.994 INFO:teuthology.orchestra.run.smithi012.stdout: debug mgr = 20 2024-04-26T04:30:47.994 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2024-04-26T04:30:47.994 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2024-04-26T04:30:47.994 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2024-04-26T04:30:47.994 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout: mgr/telemetry/nag = false 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout:[mon] 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout: debug paxos = 20 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout: mon data avail warn = 5 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout: mon mgr mkfs grace = 240 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd reporter subtree level = osd 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd prime pg temp = true 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2024-04-26T04:30:47.995 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: # 1m isn't quite enough 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: mon_down_mkfs_grace = 2m 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_filestore_osds = false 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: mon scrub interval = 300 2024-04-26T04:30:47.996 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:30:47.997 INFO:teuthology.orchestra.run.smithi012.stdout:[client] 2024-04-26T04:30:47.997 INFO:teuthology.orchestra.run.smithi012.stdout: rgw cache enabled = true 2024-04-26T04:30:47.997 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable ops log = true 2024-04-26T04:30:47.997 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable usage log = true 2024-04-26T04:30:47.997 INFO:teuthology.orchestra.run.smithi012.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T04:30:47.997 INFO:teuthology.orchestra.run.smithi012.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T04:30:47.997 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.a] 2024-04-26T04:30:47.997 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.b] 2024-04-26T04:30:47.997 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.c] 2024-04-26T04:30:47.998 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2024-04-26T04:30:47.998 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2024-04-26T04:30:47.998 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T04:30:47.998 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2024-04-26T04:30:47.998 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:47.999 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T04:30:48.000 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T04:30:48.000 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.000 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2024-04-26T04:30:48.000 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.000 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2024-04-26T04:30:48.000 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2024-04-26T04:30:48.000 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2024-04-26T04:30:48.000 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2024-04-26T04:30:48.000 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2024-04-26T04:30:48.000 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2024-04-26T04:30:48.000 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2024-04-26T04:30:48.000 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T04:30:48.000 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.12,172.21.15.113,172.21.15.137" 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: mon election default strategy = 3 2024-04-26T04:30:48.001 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug inject dispatch delay duration = 0.1 2024-04-26T04:30:48.002 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug inject dispatch delay probability = 0.1 2024-04-26T04:30:48.002 INFO:teuthology.orchestra.run.smithi178.stdout: osd_async_recovery_min_cost = 1 2024-04-26T04:30:48.002 INFO:teuthology.orchestra.run.smithi178.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-04-26T04:30:48.002 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = 12c27fbe-0161-48d0-88ec-42a226685437 2024-04-26T04:30:48.002 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.002 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2024-04-26T04:30:48.002 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2024-04-26T04:30:48.002 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.002 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2024-04-26T04:30:48.002 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 120 2024-04-26T04:30:48.002 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2024-04-26T04:30:48.002 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.002 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug trim objects = true 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2024-04-26T04:30:48.003 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: debug monc = 20 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: bdev async discard = True 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: bdev enable discard = True 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore zero block detection = True 2024-04-26T04:30:48.004 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 20 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 20 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 10 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: osd blocked scrub grace period = 3600 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug reject backfill probability = 0.1 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify cached snaps = True 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify missing on start = True 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: osd max backfills = 3 2024-04-26T04:30:48.005 INFO:teuthology.orchestra.run.smithi178.stdout: osd max markdown count = 1000 2024-04-26T04:30:48.006 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock override recovery settings = True 2024-04-26T04:30:48.006 INFO:teuthology.orchestra.run.smithi178.stdout: osd memory target = 1610612736 2024-04-26T04:30:48.006 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2024-04-26T04:30:48.006 INFO:teuthology.orchestra.run.smithi178.stdout: osd op queue = debug_random 2024-04-26T04:30:48.006 INFO:teuthology.orchestra.run.smithi178.stdout: osd op queue cut off = debug_random 2024-04-26T04:30:48.006 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub min interval = 60 2024-04-26T04:30:48.006 INFO:teuthology.orchestra.run.smithi178.stdout: osd shutdown pgref assert = True 2024-04-26T04:30:48.006 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.006 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2024-04-26T04:30:48.006 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-26T04:30:48.006 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2024-04-26T04:30:48.006 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2024-04-26T04:30:48.007 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2024-04-26T04:30:48.008 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.009 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2024-04-26T04:30:48.009 INFO:teuthology.orchestra.run.smithi178.stdout: mon scrub interval = 300 2024-04-26T04:30:48.009 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-04-26T04:30:48.009 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2024-04-26T04:30:48.009 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2024-04-26T04:30:48.009 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2024-04-26T04:30:48.009 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2024-04-26T04:30:48.009 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T04:30:48.009 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T04:30:48.009 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2024-04-26T04:30:48.009 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2024-04-26T04:30:48.009 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2024-04-26T04:30:48.009 INFO:teuthology.orchestra.run.smithi137.stdout:[global] 2024-04-26T04:30:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: chdir = "" 2024-04-26T04:30:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T04:30:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: auth supported = cephx 2024-04-26T04:30:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: filestore xattr use omap = true 2024-04-26T04:30:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: mon clock drift allowed = 1.000 2024-04-26T04:30:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: osd crush chooseleaf type = 0 2024-04-26T04:30:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: auth debug = true 2024-04-26T04:30:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on old message = true 2024-04-26T04:30:48.010 INFO:teuthology.orchestra.run.smithi137.stdout: ms die on bug = true 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: mon pg warn max object skew = 0 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: osd pool default size = 2 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd allow primary affinity = true 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd allow pg remap = true 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on legacy crush tunables = false 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on crush straw calc version zero = false 2024-04-26T04:30:48.011 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on no sortbitwise = false 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on osd down out interval zero = false 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: mon warn on too few osds = false 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: mon_allow_pool_size_one = true 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: osd default data pool replay window = 5 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: mon allow pool delete = true 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: mon cluster log file level = debug 2024-04-26T04:30:48.012 INFO:teuthology.orchestra.run.smithi137.stdout: debug asserts on shutdown = true 2024-04-26T04:30:48.013 INFO:teuthology.orchestra.run.smithi137.stdout: mon health detail to clog = false 2024-04-26T04:30:48.013 INFO:teuthology.orchestra.run.smithi137.stdout: mon host = "172.21.15.12,172.21.15.113,172.21.15.137" 2024-04-26T04:30:48.013 INFO:teuthology.orchestra.run.smithi137.stdout: mon election default strategy = 3 2024-04-26T04:30:48.013 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug inject dispatch delay duration = 0.1 2024-04-26T04:30:48.013 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug inject dispatch delay probability = 0.1 2024-04-26T04:30:48.013 INFO:teuthology.orchestra.run.smithi137.stdout: osd_async_recovery_min_cost = 1 2024-04-26T04:30:48.013 INFO:teuthology.orchestra.run.smithi137.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-04-26T04:30:48.013 INFO:teuthology.orchestra.run.smithi137.stdout: fsid = 12c27fbe-0161-48d0-88ec-42a226685437 2024-04-26T04:30:48.013 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.013 INFO:teuthology.orchestra.run.smithi137.stdout:[osd] 2024-04-26T04:30:48.013 INFO:teuthology.orchestra.run.smithi137.stdout: osd journal size = 100 2024-04-26T04:30:48.013 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.013 INFO:teuthology.orchestra.run.smithi137.stdout: osd scrub load threshold = 5.0 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: osd scrub max interval = 120 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: osd mclock profile = high_recovery_ops 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: osd recover clone overlap = true 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: osd recovery max chunk = 1048576 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug shutdown = true 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug op order = true 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug verify stray on activate = true 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug trim objects = true 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: osd open classes on start = true 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug pg log writeout = true 2024-04-26T04:30:48.014 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.017 INFO:teuthology.orchestra.run.smithi137.stdout: osd deep scrub update digest min age = 30 2024-04-26T04:30:48.017 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.017 INFO:teuthology.orchestra.run.smithi137.stdout: osd map max advance = 10 2024-04-26T04:30:48.017 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: journal zero on create = true 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: filestore ondisk finisher threads = 3 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: filestore apply finisher threads = 3 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: bdev debug aio = true 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug misdirected ops = true 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: debug monc = 20 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: bdev async discard = True 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: bdev enable discard = True 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore block size = 96636764160 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore fsck on mount = True 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: bluestore zero block detection = True 2024-04-26T04:30:48.018 INFO:teuthology.orchestra.run.smithi137.stdout: debug bluefs = 20 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: debug bluestore = 20 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: debug osd = 20 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: debug rocksdb = 10 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd full ratio = 0.9 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd nearfull ratio = 0.8 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: osd blocked scrub grace period = 3600 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug reject backfill probability = 0.1 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug verify cached snaps = True 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: osd debug verify missing on start = True 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: osd failsafe full ratio = 0.95 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: osd max backfills = 3 2024-04-26T04:30:48.019 INFO:teuthology.orchestra.run.smithi137.stdout: osd max markdown count = 1000 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout: osd mclock override recovery settings = True 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout: osd memory target = 1610612736 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout: osd objectstore = bluestore 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout: osd op queue = debug_random 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout: osd op queue cut off = debug_random 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout: osd scrub min interval = 60 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout: osd shutdown pgref assert = True 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr] 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout: debug mgr = 20 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout: debug mon = 20 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout: debug auth = 20 2024-04-26T04:30:48.020 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min pgs per osd = 4 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min bytes per osd = 10 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout: mgr/telemetry/nag = false 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout:[mon] 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout: debug ms = 1 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout: debug mon = 20 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout: debug paxos = 20 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout: debug auth = 20 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout: mon data avail warn = 5 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout: mon mgr mkfs grace = 240 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min pgs per osd = 4 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd reporter subtree level = osd 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout: mon osd prime pg temp = true 2024-04-26T04:30:48.021 INFO:teuthology.orchestra.run.smithi137.stdout: mon reweight min bytes per osd = 10 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: # 1m isn't quite enough 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: mon_down_mkfs_grace = 2m 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: mon_warn_on_filestore_osds = false 2024-04-26T04:30:48.022 INFO:teuthology.orchestra.run.smithi137.stdout: mon scrub interval = 300 2024-04-26T04:30:48.023 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-26T04:30:48.023 INFO:teuthology.orchestra.run.smithi137.stdout:[client] 2024-04-26T04:30:48.023 INFO:teuthology.orchestra.run.smithi137.stdout: rgw cache enabled = true 2024-04-26T04:30:48.023 INFO:teuthology.orchestra.run.smithi137.stdout: rgw enable ops log = true 2024-04-26T04:30:48.023 INFO:teuthology.orchestra.run.smithi137.stdout: rgw enable usage log = true 2024-04-26T04:30:48.023 INFO:teuthology.orchestra.run.smithi137.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T04:30:48.023 INFO:teuthology.orchestra.run.smithi137.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T04:30:48.023 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.a] 2024-04-26T04:30:48.023 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.b] 2024-04-26T04:30:48.023 INFO:teuthology.orchestra.run.smithi137.stdout:[mon.c] 2024-04-26T04:30:48.025 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2024-04-26T04:30:48.025 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2024-04-26T04:30:48.025 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T04:30:48.025 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2024-04-26T04:30:48.025 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.025 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2024-04-26T04:30:48.025 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.025 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2024-04-26T04:30:48.025 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.025 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2024-04-26T04:30:48.026 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2024-04-26T04:30:48.026 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.026 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2024-04-26T04:30:48.026 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2024-04-26T04:30:48.026 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.026 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T04:30:48.026 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2024-04-26T04:30:48.026 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.026 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T04:30:48.026 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T04:30:48.026 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.026 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2024-04-26T04:30:48.027 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.027 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2024-04-26T04:30:48.027 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2024-04-26T04:30:48.027 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2024-04-26T04:30:48.027 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2024-04-26T04:30:48.027 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2024-04-26T04:30:48.027 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2024-04-26T04:30:48.027 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2024-04-26T04:30:48.027 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T04:30:48.027 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T04:30:48.027 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2024-04-26T04:30:48.027 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.027 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.12,172.21.15.113,172.21.15.137" 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: mon election default strategy = 3 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug inject dispatch delay duration = 0.1 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug inject dispatch delay probability = 0.1 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: osd_async_recovery_min_cost = 1 2024-04-26T04:30:48.028 INFO:teuthology.orchestra.run.smithi113.stdout: osd_object_clean_region_max_num_intervals = 1000 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = 12c27fbe-0161-48d0-88ec-42a226685437 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 120 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2024-04-26T04:30:48.030 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2024-04-26T04:30:48.030 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2024-04-26T04:30:48.030 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug trim objects = true 2024-04-26T04:30:48.030 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.030 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2024-04-26T04:30:48.030 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2024-04-26T04:30:48.030 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.030 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2024-04-26T04:30:48.030 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.031 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2024-04-26T04:30:48.031 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.031 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2024-04-26T04:30:48.031 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.031 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2024-04-26T04:30:48.031 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2024-04-26T04:30:48.031 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.031 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2024-04-26T04:30:48.031 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: debug monc = 20 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: bdev async discard = True 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: bdev enable discard = True 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore zero block detection = True 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 20 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 20 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 10 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2024-04-26T04:30:48.032 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd blocked scrub grace period = 3600 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug reject backfill probability = 0.1 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify cached snaps = True 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify missing on start = True 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 0.95 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd max backfills = 3 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd max markdown count = 1000 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock override recovery settings = True 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd memory target = 1610612736 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = bluestore 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd op queue = debug_random 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd op queue cut off = debug_random 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub min interval = 60 2024-04-26T04:30:48.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd shutdown pgref assert = True 2024-04-26T04:30:48.034 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.034 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2024-04-26T04:30:48.034 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-26T04:30:48.034 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2024-04-26T04:30:48.034 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-26T04:30:48.034 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-26T04:30:48.034 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-26T04:30:48.034 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-26T04:30:48.034 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2024-04-26T04:30:48.034 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.034 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2024-04-26T04:30:48.035 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-04-26T04:30:48.035 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-04-26T04:30:48.035 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2024-04-26T04:30:48.035 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-04-26T04:30:48.035 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2024-04-26T04:30:48.035 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2024-04-26T04:30:48.035 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-04-26T04:30:48.035 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2024-04-26T04:30:48.035 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2024-04-26T04:30:48.035 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-04-26T04:30:48.035 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.035 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T04:30:48.036 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T04:30:48.036 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T04:30:48.036 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.036 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T04:30:48.036 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T04:30:48.036 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T04:30:48.036 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.036 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2024-04-26T04:30:48.036 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2024-04-26T04:30:48.036 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.036 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2024-04-26T04:30:48.036 INFO:teuthology.orchestra.run.smithi113.stdout: mon scrub interval = 300 2024-04-26T04:30:48.037 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-04-26T04:30:48.037 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2024-04-26T04:30:48.037 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2024-04-26T04:30:48.037 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2024-04-26T04:30:48.037 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2024-04-26T04:30:48.037 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T04:30:48.037 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T04:30:48.037 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2024-04-26T04:30:48.037 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2024-04-26T04:30:48.037 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2024-04-26T04:30:48.039 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T04:30:48.039 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T04:30:48.117 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T04:30:48.118 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:30:48.118 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T04:30:48.123 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:30:48.124 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T04:30:48.171 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T04:30:48.171 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:30:48.171 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T04:30:48.171 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T04:30:48.236 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:30:48.236 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:30:48.283 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi113.front.sepia.ceph.com 2024-04-26T04:30:48.283 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:30:48.283 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T04:30:48.283 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T04:30:48.307 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:30:48.307 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:30:48.355 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi137.front.sepia.ceph.com 2024-04-26T04:30:48.355 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:30:48.356 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T04:30:48.356 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T04:30:48.379 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:30:48.379 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:30:48.430 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi178.front.sepia.ceph.com 2024-04-26T04:30:48.430 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:30:48.431 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T04:30:48.431 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T04:30:48.454 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:30:48.454 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:30:48.503 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T04:30:48.503 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T04:30:48.503 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T04:30:48.549 INFO:teuthology.orchestra.run.smithi012.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T04:30:48.552 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T04:30:48.595 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T04:30:48.612 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T04:30:48.612 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T04:30:48.612 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T04:30:48.684 INFO:teuthology.orchestra.run.smithi178.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T04:30:48.693 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T04:30:48.693 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi137.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-04-26T04:30:48.693 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T04:30:48.708 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-26T04:30:48.709 INFO:tasks.ceph:role: osd.0 2024-04-26T04:30:48.710 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T04:30:48.710 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T04:30:48.776 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:30:48.776 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:30:48.776 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:30:48.776 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:30:48.776 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:30:48.776 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:30:48.776 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:30:48.777 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:30:48.777 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:30:48.777 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:30:49.247 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2024-04-26T04:30:49.247 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-04-26T04:30:49.247 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T04:30:49.342 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T04:30:49.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:30:49.353 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:30:49.353 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T04:30:49.404 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-26T04:30:49.404 INFO:tasks.ceph:role: osd.4 2024-04-26T04:30:49.404 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T04:30:49.404 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T04:30:49.466 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:30:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:30:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:30:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:30:49.466 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:30:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:30:49.466 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:30:49.466 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:30:49.466 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:30:49.467 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:30:49.937 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2024-04-26T04:30:49.939 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-04-26T04:30:49.940 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-04-26T04:30:50.091 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T04:30:50.102 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:30:50.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:30:50.104 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2024-04-26T04:30:50.159 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-26T04:30:50.159 INFO:tasks.ceph:role: osd.8 2024-04-26T04:30:50.159 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T04:30:50.159 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T04:30:50.223 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:30:50.223 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:30:50.223 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:30:50.223 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:30:50.223 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:30:50.223 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:30:50.223 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:30:50.224 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:30:50.224 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:30:50.224 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:30:50.687 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2024-04-26T04:30:50.689 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-04-26T04:30:50.689 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-8 2024-04-26T04:30:50.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2024-04-26T04:30:50.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:30:50.853 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:30:50.853 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-12 2024-04-26T04:30:50.907 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-26T04:30:50.908 INFO:tasks.ceph:role: osd.12 2024-04-26T04:30:50.908 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T04:30:50.908 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T04:30:50.971 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:30:50.971 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:30:50.971 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:30:50.971 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:30:50.971 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:30:50.971 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:30:50.971 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:30:50.971 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:30:50.972 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:30:50.972 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:30:51.408 INFO:teuthology.orchestra.run.smithi012.stdout:Discarding blocks...Done. 2024-04-26T04:30:51.410 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-04-26T04:30:51.411 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-12 2024-04-26T04:30:51.501 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-12 2024-04-26T04:30:51.513 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:30:51.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:30:51.514 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:30:51.623 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:51.620+0000 7f3142e0e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T04:30:51.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:51.624+0000 7f3142e0e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T04:30:51.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:51.624+0000 7f3142e0e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T04:30:51.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:51.624+0000 7f3142e0e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T04:30:51.627 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:51.624+0000 7f3142e0e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T04:30:51.628 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:51.624+0000 7f3142e0e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T04:30:54.612 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T04:30:54.626 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:30:54.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:54.708+0000 7f0d588d17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T04:30:54.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:54.708+0000 7f0d588d17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T04:30:54.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:54.708+0000 7f0d588d17c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T04:30:54.712 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:54.708+0000 7f0d588d17c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T04:30:54.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:54.708+0000 7f0d588d17c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T04:30:54.713 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:54.708+0000 7f0d588d17c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T04:30:57.722 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T04:30:57.780 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:30:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:57.860+0000 7fd4b7aab7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2024-04-26T04:30:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:57.860+0000 7fd4b7aab7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2024-04-26T04:30:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:57.860+0000 7fd4b7aab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-04-26T04:30:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:57.860+0000 7fd4b7aab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-04-26T04:30:57.863 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:57.860+0000 7fd4b7aab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-04-26T04:30:57.864 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:30:57.860+0000 7fd4b7aab7c0 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2024-04-26T04:31:00.853 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2024-04-26T04:31:00.915 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 12 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:00.998 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:31:00.992+0000 7fdea41b07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-12/keyring: can't open /var/lib/ceph/osd/ceph-12/keyring: (2) No such file or directory 2024-04-26T04:31:00.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:31:00.992+0000 7fdea41b07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-12/keyring 2024-04-26T04:31:00.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:31:00.992+0000 7fdea41b07c0 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-04-26T04:31:00.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:31:00.992+0000 7fdea41b07c0 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-04-26T04:31:00.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:31:00.992+0000 7fdea41b07c0 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-04-26T04:31:00.999 INFO:teuthology.orchestra.run.smithi012.stderr:2024-04-26T04:31:00.992+0000 7fdea41b07c0 -1 bluestore(/var/lib/ceph/osd/ceph-12) _read_fsid unparsable uuid 2024-04-26T04:31:03.970 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-12 2024-04-26T04:31:04.028 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T04:31:04.043 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-26T04:31:04.043 INFO:tasks.ceph:role: osd.1 2024-04-26T04:31:04.043 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-26T04:31:04.043 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T04:31:04.106 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:31:04.107 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:31:04.107 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:31:04.107 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:31:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:31:04.107 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:31:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:31:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:31:04.107 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:31:04.107 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:31:04.605 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-26T04:31:04.607 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-26T04:31:04.607 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-04-26T04:31:04.734 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T04:31:04.745 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:31:04.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:31:04.747 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T04:31:04.803 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-26T04:31:04.803 INFO:tasks.ceph:role: osd.5 2024-04-26T04:31:04.803 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-26T04:31:04.803 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T04:31:04.870 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:31:04.870 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:31:04.870 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:31:04.871 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:31:04.871 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:31:04.871 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:31:04.871 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:31:04.871 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:31:04.871 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:31:04.871 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:31:05.377 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-26T04:31:05.379 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-26T04:31:05.379 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T04:31:05.481 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T04:31:05.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:31:05.492 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:31:05.492 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2024-04-26T04:31:05.545 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-26T04:31:05.545 INFO:tasks.ceph:role: osd.9 2024-04-26T04:31:05.545 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-26T04:31:05.545 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T04:31:05.606 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:31:05.607 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:31:05.607 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:31:05.607 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:31:05.607 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:31:05.608 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:31:05.608 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:31:05.608 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:31:05.608 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:31:05.608 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:31:06.093 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-26T04:31:06.094 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-26T04:31:06.095 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9 2024-04-26T04:31:06.208 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2024-04-26T04:31:06.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:31:06.218 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:31:06.219 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-13 2024-04-26T04:31:06.274 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-26T04:31:06.275 INFO:tasks.ceph:role: osd.13 2024-04-26T04:31:06.275 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-26T04:31:06.275 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T04:31:06.337 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:31:06.337 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:31:06.337 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:31:06.337 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:31:06.337 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:31:06.337 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:31:06.337 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:31:06.337 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:31:06.337 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:31:06.338 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:31:06.863 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-04-26T04:31:06.866 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-04-26T04:31:06.866 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-13 2024-04-26T04:31:06.995 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-13 2024-04-26T04:31:07.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:31:07.008 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:31:07.009 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:07.122 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:07.121+0000 7fa4fde987c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T04:31:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:07.125+0000 7fa4fde987c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T04:31:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:07.125+0000 7fa4fde987c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T04:31:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:07.125+0000 7fa4fde987c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T04:31:07.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:07.125+0000 7fa4fde987c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T04:31:07.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:07.133+0000 7fa4fde987c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T04:31:10.114 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T04:31:10.131 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:10.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:10.213+0000 7f11af3977c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T04:31:10.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:10.213+0000 7f11af3977c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T04:31:10.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:10.213+0000 7f11af3977c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T04:31:10.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:10.213+0000 7f11af3977c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T04:31:10.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:10.213+0000 7f11af3977c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T04:31:10.213 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:10.213+0000 7f11af3977c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T04:31:13.195 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T04:31:13.252 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:13.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:13.333+0000 7f19e573f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2024-04-26T04:31:13.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:13.333+0000 7f19e573f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2024-04-26T04:31:13.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:13.333+0000 7f19e573f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-04-26T04:31:13.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:13.333+0000 7f19e573f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-04-26T04:31:13.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:13.333+0000 7f19e573f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-04-26T04:31:13.334 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:13.333+0000 7f19e573f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2024-04-26T04:31:16.297 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2024-04-26T04:31:16.352 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 13 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:16.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:16.433+0000 7fc87baec7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-13/keyring: can't open /var/lib/ceph/osd/ceph-13/keyring: (2) No such file or directory 2024-04-26T04:31:16.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:16.433+0000 7fc87baec7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-13/keyring 2024-04-26T04:31:16.432 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:16.433+0000 7fc87baec7c0 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-04-26T04:31:16.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:16.433+0000 7fc87baec7c0 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-04-26T04:31:16.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:16.433+0000 7fc87baec7c0 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-04-26T04:31:16.433 INFO:teuthology.orchestra.run.smithi113.stderr:2024-04-26T04:31:16.433+0000 7fc87baec7c0 -1 bluestore(/var/lib/ceph/osd/ceph-13) _read_fsid unparsable uuid 2024-04-26T04:31:19.430 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-13 2024-04-26T04:31:19.487 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T04:31:19.502 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-26T04:31:19.502 INFO:tasks.ceph:role: osd.2 2024-04-26T04:31:19.502 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-26T04:31:19.502 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T04:31:19.567 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:31:19.567 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:31:19.567 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:31:19.567 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:31:19.567 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:31:19.567 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:31:19.567 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:31:19.567 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:31:19.567 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:31:19.567 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:31:20.015 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-04-26T04:31:20.017 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-04-26T04:31:20.017 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-04-26T04:31:20.110 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T04:31:20.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:31:20.120 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:31:20.120 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T04:31:20.172 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-26T04:31:20.173 INFO:tasks.ceph:role: osd.6 2024-04-26T04:31:20.173 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-26T04:31:20.173 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T04:31:20.234 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:31:20.234 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:31:20.234 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:31:20.234 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:31:20.234 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:31:20.235 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:31:20.235 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:31:20.235 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:31:20.235 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:31:20.235 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:31:20.706 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-04-26T04:31:20.708 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-04-26T04:31:20.708 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2024-04-26T04:31:20.848 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T04:31:20.859 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:31:20.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:31:20.860 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2024-04-26T04:31:20.914 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-26T04:31:20.914 INFO:tasks.ceph:role: osd.10 2024-04-26T04:31:20.914 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-26T04:31:20.914 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T04:31:20.978 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:31:20.978 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:31:20.978 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:31:20.978 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:31:20.979 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:31:20.979 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:31:20.979 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:31:20.979 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:31:20.979 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:31:20.979 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:31:21.459 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-04-26T04:31:21.463 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-04-26T04:31:21.463 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2024-04-26T04:31:21.607 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2024-04-26T04:31:21.618 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:31:21.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:31:21.619 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/osd/ceph-14 2024-04-26T04:31:21.674 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-26T04:31:21.674 INFO:tasks.ceph:role: osd.14 2024-04-26T04:31:21.674 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-26T04:31:21.675 DEBUG:teuthology.orchestra.run.smithi137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T04:31:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:31:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:31:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:31:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:31:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:31:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:31:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:31:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:31:21.739 INFO:teuthology.orchestra.run.smithi137.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:31:21.739 INFO:teuthology.orchestra.run.smithi137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:31:22.159 INFO:teuthology.orchestra.run.smithi137.stdout:Discarding blocks...Done. 2024-04-26T04:31:22.162 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi137.front.sepia.ceph.com -o noatime 2024-04-26T04:31:22.162 DEBUG:teuthology.orchestra.run.smithi137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-14 2024-04-26T04:31:22.252 DEBUG:teuthology.orchestra.run.smithi137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-14 2024-04-26T04:31:22.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:31:22.262 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:31:22.262 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:22.369 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:22.365+0000 7fe297e3e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T04:31:22.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:22.369+0000 7fe297e3e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T04:31:22.370 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:22.369+0000 7fe297e3e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T04:31:22.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:22.369+0000 7fe297e3e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T04:31:22.371 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:22.369+0000 7fe297e3e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T04:31:22.377 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:22.373+0000 7fe297e3e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T04:31:25.390 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T04:31:25.404 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:25.483 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:25.481+0000 7f17754357c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T04:31:25.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:25.481+0000 7f17754357c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T04:31:25.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:25.481+0000 7f17754357c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T04:31:25.484 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:25.481+0000 7f17754357c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T04:31:25.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:25.481+0000 7f17754357c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T04:31:25.485 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:25.481+0000 7f17754357c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T04:31:28.445 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T04:31:28.498 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:28.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:28.577+0000 7f0dc16887c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2024-04-26T04:31:28.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:28.577+0000 7f0dc16887c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2024-04-26T04:31:28.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:28.577+0000 7f0dc16887c0 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-04-26T04:31:28.580 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:28.577+0000 7f0dc16887c0 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-04-26T04:31:28.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:28.577+0000 7f0dc16887c0 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-04-26T04:31:28.581 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:28.577+0000 7f0dc16887c0 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2024-04-26T04:31:31.556 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2024-04-26T04:31:31.612 DEBUG:teuthology.orchestra.run.smithi137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 14 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:31.693 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:31.689+0000 7f68b56b97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-14/keyring: can't open /var/lib/ceph/osd/ceph-14/keyring: (2) No such file or directory 2024-04-26T04:31:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:31.689+0000 7f68b56b97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-14/keyring 2024-04-26T04:31:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:31.689+0000 7f68b56b97c0 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-04-26T04:31:31.694 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:31.689+0000 7f68b56b97c0 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-04-26T04:31:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:31.689+0000 7f68b56b97c0 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-04-26T04:31:31.695 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-26T04:31:31.689+0000 7f68b56b97c0 -1 bluestore(/var/lib/ceph/osd/ceph-14) _read_fsid unparsable uuid 2024-04-26T04:31:34.677 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-14 2024-04-26T04:31:34.731 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T04:31:34.746 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-26T04:31:34.746 INFO:tasks.ceph:role: osd.3 2024-04-26T04:31:34.746 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-26T04:31:34.746 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T04:31:34.802 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:31:34.803 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:31:34.803 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:31:34.803 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:31:34.803 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:31:34.803 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:31:34.803 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:31:34.803 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:31:34.803 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:31:34.803 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:31:34.826 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-04-26T04:31:34.828 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-04-26T04:31:34.828 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-04-26T04:31:34.886 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T04:31:34.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:31:34.939 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:31:34.939 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T04:31:34.992 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-26T04:31:34.992 INFO:tasks.ceph:role: osd.7 2024-04-26T04:31:34.992 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-26T04:31:34.992 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T04:31:35.050 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:31:35.050 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:31:35.050 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:31:35.050 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:31:35.050 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:31:35.050 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:31:35.050 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:31:35.050 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:31:35.051 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:31:35.051 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:31:35.074 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-04-26T04:31:35.075 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-04-26T04:31:35.075 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-7 2024-04-26T04:31:35.135 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T04:31:35.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:31:35.187 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:31:35.187 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2024-04-26T04:31:35.239 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-26T04:31:35.240 INFO:tasks.ceph:role: osd.11 2024-04-26T04:31:35.240 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-26T04:31:35.240 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T04:31:35.293 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:31:35.293 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:31:35.293 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:31:35.293 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:31:35.293 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:31:35.294 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:31:35.294 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:31:35.294 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:31:35.294 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:31:35.294 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:31:35.317 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-04-26T04:31:35.319 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-04-26T04:31:35.319 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-11 2024-04-26T04:31:35.378 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2024-04-26T04:31:35.432 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:31:35.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:31:35.433 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-15 2024-04-26T04:31:35.486 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-26T04:31:35.486 INFO:tasks.ceph:role: osd.15 2024-04-26T04:31:35.486 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-26T04:31:35.486 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T04:31:35.542 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T04:31:35.543 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T04:31:35.543 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T04:31:35.543 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-26T04:31:35.543 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T04:31:35.543 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2024-04-26T04:31:35.543 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T04:31:35.543 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-26T04:31:35.543 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T04:31:35.543 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T04:31:35.566 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2024-04-26T04:31:35.568 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2024-04-26T04:31:35.568 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-15 2024-04-26T04:31:35.630 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-15 2024-04-26T04:31:35.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:31:35.684 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: /sbin/restorecon: command not found 2024-04-26T04:31:35.684 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:35.795 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:35.793+0000 7f41e06fd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T04:31:35.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:35.793+0000 7f41e06fd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T04:31:35.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:35.793+0000 7f41e06fd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T04:31:35.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:35.793+0000 7f41e06fd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T04:31:35.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:35.793+0000 7f41e06fd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T04:31:35.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:35.793+0000 7f41e06fd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T04:31:38.783 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T04:31:38.796 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:38.880 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:38.877+0000 7f63410157c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T04:31:38.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:38.877+0000 7f63410157c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T04:31:38.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:38.877+0000 7f63410157c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T04:31:38.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:38.877+0000 7f63410157c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T04:31:38.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:38.877+0000 7f63410157c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T04:31:38.881 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:38.877+0000 7f63410157c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T04:31:41.845 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T04:31:41.899 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:41.981 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:41.977+0000 7fbd61dbc7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2024-04-26T04:31:41.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:41.977+0000 7fbd61dbc7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2024-04-26T04:31:41.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:41.977+0000 7fbd61dbc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-04-26T04:31:41.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:41.977+0000 7fbd61dbc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-04-26T04:31:41.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:41.977+0000 7fbd61dbc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-04-26T04:31:41.982 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:41.977+0000 7fbd61dbc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2024-04-26T04:31:44.973 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2024-04-26T04:31:45.030 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 15 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:45.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:45.109+0000 7f12199047c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-15/keyring: can't open /var/lib/ceph/osd/ceph-15/keyring: (2) No such file or directory 2024-04-26T04:31:45.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:45.109+0000 7f12199047c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-15/keyring 2024-04-26T04:31:45.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:45.109+0000 7f12199047c0 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-04-26T04:31:45.113 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:45.109+0000 7f12199047c0 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-04-26T04:31:45.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:45.109+0000 7f12199047c0 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-04-26T04:31:45.114 INFO:teuthology.orchestra.run.smithi178.stderr:2024-04-26T04:31:45.109+0000 7f12199047c0 -1 bluestore(/var/lib/ceph/osd/ceph-15) _read_fsid unparsable uuid 2024-04-26T04:31:48.114 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-15 2024-04-26T04:31:48.171 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T04:31:48.171 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:31:48.172 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T04:31:48.186 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:31:48.187 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T04:31:48.243 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:31:48.243 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T04:31:48.298 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:31:48.298 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2024-04-26T04:31:48.354 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:31:48.354 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-12/keyring of=/dev/stdout 2024-04-26T04:31:48.410 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:31:48.410 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T04:31:48.424 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:31:48.424 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T04:31:48.479 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:31:48.479 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2024-04-26T04:31:48.540 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:31:48.540 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-13/keyring of=/dev/stdout 2024-04-26T04:31:48.594 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:31:48.594 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T04:31:48.608 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:31:48.608 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T04:31:48.662 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:31:48.662 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2024-04-26T04:31:48.717 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:31:48.717 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-14/keyring of=/dev/stdout 2024-04-26T04:31:48.770 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:31:48.770 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T04:31:48.783 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:31:48.784 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T04:31:48.838 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:31:48.838 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T04:31:48.893 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:31:48.893 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2024-04-26T04:31:48.949 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:31:48.949 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-15/keyring of=/dev/stdout 2024-04-26T04:31:49.001 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:31:49.001 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T04:31:49.046 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T04:31:49.046 DEBUG:teuthology.orchestra.run.smithi012:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T04:31:49.049 DEBUG:teuthology.orchestra.run.smithi113:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T04:31:49.051 DEBUG:teuthology.orchestra.run.smithi137:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T04:31:49.059 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr.y] 2024-04-26T04:31:49.059 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQD4LStm4zWkIBAAvlJyYBvEKUZprOWk/BQzPA== 2024-04-26T04:31:49.059 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.0] 2024-04-26T04:31:49.059 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQD7LStmFzANJRAA21I7EV67KFUDlEM8e1bp/g== 2024-04-26T04:31:49.059 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.4] 2024-04-26T04:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQD+LStmuyNRKhAA+boFzGDxekHRqVOZwQPvSg== 2024-04-26T04:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.8] 2024-04-26T04:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQABLitm9pVYMxAAP2FK8L8GUkY3LnqyQ2YqSw== 2024-04-26T04:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.12] 2024-04-26T04:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQAELitm10ZbOxAApn1n3+XxSc52fW3/5uPH3A== 2024-04-26T04:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.1] 2024-04-26T04:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQALLitmG6aABxAAVAIqL2F/Y1T7qKjY0+KwcA== 2024-04-26T04:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.5] 2024-04-26T04:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQAOLitmribWDBAAu8YFw3Kf8xiAtPO5du1Mvw== 2024-04-26T04:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.9] 2024-04-26T04:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQARLitm53cNFBAAFLW/nHpSS4v5Mv9v9wUYKQ== 2024-04-26T04:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.13] 2024-04-26T04:31:49.060 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQAULitmxaHxGRAAPDuPRAG9TnDi6bAUWHhPag== 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.2] 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQAaLitm9RAhFhAA1MFT48gUk6drqJsC7jzGug== 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.6] 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQAdLitmOGztHBAA4zTfNinn8xcy8UYwjWN+zg== 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.10] 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQAgLitm2pepIhAAidrGARm2rOgU1wHMldW2HQ== 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.14] 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQAjLitm2n9kKRAAGaSxkRpuhBwr+1quDPWxSw== 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr.x] 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQD4LStmINaUIxAAmMWA1hbdAQJuCuUfz29bPw== 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.3] 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQAnLitmaZJ7LxAAyL77xwm5BRW306SQoev+8g== 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.7] 2024-04-26T04:31:49.061 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQAqLitm4HeRNBAAmodhRPCDgM62BctCthwYnA== 2024-04-26T04:31:49.062 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.11] 2024-04-26T04:31:49.062 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQAtLitmDn6QOhAAOdD4Flinq+Gtj18LUdynug== 2024-04-26T04:31:49.062 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.15] 2024-04-26T04:31:49.062 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQAxLitmoZ7TBhAAswJL9hIrVoHlWpEZwiZoxg== 2024-04-26T04:31:49.062 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-04-26T04:31:49.062 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQD4LStms2beKBAA8EWrlgxhreLfJdlRpZi/IA== 2024-04-26T04:31:49.063 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.y] 2024-04-26T04:31:49.064 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD4LStm4zWkIBAAvlJyYBvEKUZprOWk/BQzPA== 2024-04-26T04:31:49.064 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.0] 2024-04-26T04:31:49.064 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD7LStmFzANJRAA21I7EV67KFUDlEM8e1bp/g== 2024-04-26T04:31:49.064 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.4] 2024-04-26T04:31:49.064 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD+LStmuyNRKhAA+boFzGDxekHRqVOZwQPvSg== 2024-04-26T04:31:49.064 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.8] 2024-04-26T04:31:49.065 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQABLitm9pVYMxAAP2FK8L8GUkY3LnqyQ2YqSw== 2024-04-26T04:31:49.065 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.12] 2024-04-26T04:31:49.065 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAELitm10ZbOxAApn1n3+XxSc52fW3/5uPH3A== 2024-04-26T04:31:49.065 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.1] 2024-04-26T04:31:49.065 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQALLitmG6aABxAAVAIqL2F/Y1T7qKjY0+KwcA== 2024-04-26T04:31:49.065 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.5] 2024-04-26T04:31:49.065 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAOLitmribWDBAAu8YFw3Kf8xiAtPO5du1Mvw== 2024-04-26T04:31:49.065 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.9] 2024-04-26T04:31:49.065 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQARLitm53cNFBAAFLW/nHpSS4v5Mv9v9wUYKQ== 2024-04-26T04:31:49.065 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.13] 2024-04-26T04:31:49.065 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAULitmxaHxGRAAPDuPRAG9TnDi6bAUWHhPag== 2024-04-26T04:31:49.065 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.2] 2024-04-26T04:31:49.065 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAaLitm9RAhFhAA1MFT48gUk6drqJsC7jzGug== 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.6] 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAdLitmOGztHBAA4zTfNinn8xcy8UYwjWN+zg== 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.10] 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAgLitm2pepIhAAidrGARm2rOgU1wHMldW2HQ== 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.14] 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAjLitm2n9kKRAAGaSxkRpuhBwr+1quDPWxSw== 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.x] 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD4LStmINaUIxAAmMWA1hbdAQJuCuUfz29bPw== 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.3] 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAnLitmaZJ7LxAAyL77xwm5BRW306SQoev+8g== 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.7] 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAqLitm4HeRNBAAmodhRPCDgM62BctCthwYnA== 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.11] 2024-04-26T04:31:49.066 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAtLitmDn6QOhAAOdD4Flinq+Gtj18LUdynug== 2024-04-26T04:31:49.067 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.15] 2024-04-26T04:31:49.067 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAxLitmoZ7TBhAAswJL9hIrVoHlWpEZwiZoxg== 2024-04-26T04:31:49.067 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-04-26T04:31:49.067 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQD4LStms2beKBAA8EWrlgxhreLfJdlRpZi/IA== 2024-04-26T04:31:49.068 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr.y] 2024-04-26T04:31:49.068 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD4LStm4zWkIBAAvlJyYBvEKUZprOWk/BQzPA== 2024-04-26T04:31:49.068 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.0] 2024-04-26T04:31:49.068 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD7LStmFzANJRAA21I7EV67KFUDlEM8e1bp/g== 2024-04-26T04:31:49.068 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.4] 2024-04-26T04:31:49.068 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD+LStmuyNRKhAA+boFzGDxekHRqVOZwQPvSg== 2024-04-26T04:31:49.068 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.8] 2024-04-26T04:31:49.068 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQABLitm9pVYMxAAP2FK8L8GUkY3LnqyQ2YqSw== 2024-04-26T04:31:49.068 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.12] 2024-04-26T04:31:49.068 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAELitm10ZbOxAApn1n3+XxSc52fW3/5uPH3A== 2024-04-26T04:31:49.068 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.1] 2024-04-26T04:31:49.068 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQALLitmG6aABxAAVAIqL2F/Y1T7qKjY0+KwcA== 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.5] 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAOLitmribWDBAAu8YFw3Kf8xiAtPO5du1Mvw== 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.9] 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQARLitm53cNFBAAFLW/nHpSS4v5Mv9v9wUYKQ== 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.13] 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAULitmxaHxGRAAPDuPRAG9TnDi6bAUWHhPag== 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.2] 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAaLitm9RAhFhAA1MFT48gUk6drqJsC7jzGug== 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.6] 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAdLitmOGztHBAA4zTfNinn8xcy8UYwjWN+zg== 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.10] 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAgLitm2pepIhAAidrGARm2rOgU1wHMldW2HQ== 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.14] 2024-04-26T04:31:49.069 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAjLitm2n9kKRAAGaSxkRpuhBwr+1quDPWxSw== 2024-04-26T04:31:49.070 INFO:teuthology.orchestra.run.smithi137.stdout:[mgr.x] 2024-04-26T04:31:49.070 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD4LStmINaUIxAAmMWA1hbdAQJuCuUfz29bPw== 2024-04-26T04:31:49.070 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.3] 2024-04-26T04:31:49.070 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAnLitmaZJ7LxAAyL77xwm5BRW306SQoev+8g== 2024-04-26T04:31:49.070 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.7] 2024-04-26T04:31:49.070 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAqLitm4HeRNBAAmodhRPCDgM62BctCthwYnA== 2024-04-26T04:31:49.070 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.11] 2024-04-26T04:31:49.070 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAtLitmDn6QOhAAOdD4Flinq+Gtj18LUdynug== 2024-04-26T04:31:49.070 INFO:teuthology.orchestra.run.smithi137.stdout:[osd.15] 2024-04-26T04:31:49.070 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQAxLitmoZ7TBhAAswJL9hIrVoHlWpEZwiZoxg== 2024-04-26T04:31:49.071 INFO:teuthology.orchestra.run.smithi137.stdout:[client.0] 2024-04-26T04:31:49.071 INFO:teuthology.orchestra.run.smithi137.stdout: key = AQD4LStms2beKBAA8EWrlgxhreLfJdlRpZi/IA== 2024-04-26T04:31:49.071 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T04:31:49.109 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T04:31:49.112 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T04:31:49.159 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.162 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.164 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.246 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.249 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.252 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.330 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.336 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.338 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.414 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.420 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.424 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.497 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.504 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.508 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.577 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.588 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.590 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.663 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.665 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.676 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.718 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.748 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.764 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.805 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.832 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.848 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.886 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.916 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.932 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.978 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:49.981 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.020 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.065 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.067 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.104 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.150 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T04:31:50.156 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T04:31:50.183 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T04:31:50.232 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.240 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.268 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.317 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.324 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.348 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.399 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.402 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.428 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.482 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.484 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.512 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T04:31:50.568 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T04:31:50.570 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T04:31:50.595 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T04:31:50.652 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T04:31:50.652 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T04:31:50.666 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T04:31:50.758 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T04:31:50.769 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T04:31:50.783 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T04:31:50.879 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T04:31:50.890 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T04:31:50.904 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T04:31:50.996 DEBUG:teuthology.orchestra.run.smithi137:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T04:31:51.007 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:51.010 DEBUG:teuthology.orchestra.run.smithi113:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:51.012 DEBUG:teuthology.orchestra.run.smithi137:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T04:31:51.054 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T04:31:51.055 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T04:31:51.055 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T04:31:51.057 INFO:tasks.ceph.mon.a:Started 2024-04-26T04:31:51.057 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T04:31:51.057 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T04:31:51.059 INFO:tasks.ceph.mon.b:Started 2024-04-26T04:31:51.059 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T04:31:51.060 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T04:31:51.100 INFO:tasks.ceph.mon.c:Started 2024-04-26T04:31:51.100 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T04:31:51.100 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T04:31:51.101 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T04:31:51.103 INFO:tasks.ceph.mgr.y:Started 2024-04-26T04:31:51.103 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T04:31:51.103 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T04:31:51.106 INFO:tasks.ceph.mgr.x:Started 2024-04-26T04:31:51.106 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T04:31:51.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T04:31:56.574 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-04-26T04:31:56.594 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T04:31:56.595 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T04:31:56.595 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:31:56.595 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T04:31:56.605 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:31:56.605 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T04:31:56.661 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:31:56.661 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2024-04-26T04:31:56.714 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-04-26T04:31:56.715 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-12/fsid of=/dev/stdout 2024-04-26T04:31:56.738 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:56.732+0000 7f2f6b1af180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T04:31:56.750 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:31:56.750 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T04:31:56.764 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:31:56.764 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T04:31:56.818 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:31:56.818 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2024-04-26T04:31:56.823 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:56.820+0000 7f45781a1180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T04:31:56.873 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-04-26T04:31:56.874 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-13/fsid of=/dev/stdout 2024-04-26T04:31:56.926 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:31:56.926 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T04:31:56.940 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:31:56.940 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T04:31:56.994 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:31:56.994 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2024-04-26T04:31:57.052 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-26T04:31:57.052 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/var/lib/ceph/osd/ceph-14/fsid of=/dev/stdout 2024-04-26T04:31:57.106 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:31:57.106 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T04:31:57.120 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:31:57.120 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T04:31:57.174 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:31:57.174 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2024-04-26T04:31:57.230 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-04-26T04:31:57.231 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-15/fsid of=/dev/stdout 2024-04-26T04:31:57.240 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:57.236+0000 7f2f6b1af180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T04:31:57.286 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 55a4f254-2c2a-4007-9c52-b89df6fffb69 0 2024-04-26T04:31:57.367 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:57.364+0000 7f45781a1180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T04:31:57.443 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:57.440+0000 7f2f6b1af180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T04:31:57.550 INFO:teuthology.orchestra.run.smithi178.stdout:0 2024-04-26T04:31:57.566 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 32db4f97-c8ac-481c-9e80-18fc91004298 1 2024-04-26T04:31:57.568 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:57.564+0000 7f45781a1180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T04:31:57.772 INFO:teuthology.orchestra.run.smithi178.stdout:1 2024-04-26T04:31:57.791 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 2cd26466-5d59-4599-ba3f-a78b0e3b681f 2 2024-04-26T04:31:58.002 INFO:teuthology.orchestra.run.smithi178.stdout:2 2024-04-26T04:31:58.011 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:58.008+0000 7f2f6b1af180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T04:31:58.020 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 0945017d-9525-49d6-aa67-6635acd94c3e 3 2024-04-26T04:31:58.068 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:58.064+0000 7f2f6b1af180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T04:31:58.123 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:58.120+0000 7f2f6b1af180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T04:31:58.143 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:58.140+0000 7f45781a1180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T04:31:58.198 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:58.192+0000 7f45781a1180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T04:31:58.204 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:58.200+0000 7f2f6b1af180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T04:31:58.215 INFO:teuthology.orchestra.run.smithi178.stdout:3 2024-04-26T04:31:58.232 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 38dc68bf-0a89-4900-9d35-24c70368447a 4 2024-04-26T04:31:58.252 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:58.248+0000 7f45781a1180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T04:31:58.259 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:58.256+0000 7f2f6b1af180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T04:31:58.331 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:58.328+0000 7f45781a1180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T04:31:58.340 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:58.336+0000 7f2f6b1af180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T04:31:58.385 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:58.380+0000 7f45781a1180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T04:31:58.397 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:58.392+0000 7f2f6b1af180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T04:31:58.429 INFO:teuthology.orchestra.run.smithi178.stdout:4 2024-04-26T04:31:58.446 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new e91cc8a1-57e1-4760-a329-8a60aa752a95 5 2024-04-26T04:31:58.463 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:58.460+0000 7f45781a1180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T04:31:58.518 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:58.516+0000 7f45781a1180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T04:31:58.554 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:58.552+0000 7f2f6b1af180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T04:31:58.619 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:58.616+0000 7f2f6b1af180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T04:31:58.659 INFO:teuthology.orchestra.run.smithi178.stdout:5 2024-04-26T04:31:58.671 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:58.668+0000 7f45781a1180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T04:31:58.675 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new a1322130-3391-432d-a1ee-f83297885029 6 2024-04-26T04:31:58.734 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:58.732+0000 7f45781a1180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T04:31:58.872 INFO:teuthology.orchestra.run.smithi178.stdout:6 2024-04-26T04:31:58.888 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 02ee4ca2-9d48-4aff-9641-1823760c7a3b 7 2024-04-26T04:31:59.123 INFO:teuthology.orchestra.run.smithi178.stdout:7 2024-04-26T04:31:59.138 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new ea3d5f62-3d68-4824-8556-da8db36553fb 8 2024-04-26T04:31:59.276 INFO:tasks.ceph.mgr.y.smithi012.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-26T04:31:59.276 INFO:tasks.ceph.mgr.y.smithi012.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-26T04:31:59.276 INFO:tasks.ceph.mgr.y.smithi012.stderr: from numpy import show_config as show_numpy_config 2024-04-26T04:31:59.330 INFO:teuthology.orchestra.run.smithi178.stdout:8 2024-04-26T04:31:59.331 INFO:tasks.ceph.mgr.x.smithi178.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-26T04:31:59.331 INFO:tasks.ceph.mgr.x.smithi178.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-26T04:31:59.331 INFO:tasks.ceph.mgr.x.smithi178.stderr: from numpy import show_config as show_numpy_config 2024-04-26T04:31:59.346 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70 9 2024-04-26T04:31:59.355 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:59.352+0000 7f2f6b1af180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T04:31:59.413 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:59.408+0000 7f45781a1180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T04:31:59.543 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:59.540+0000 7f2f6b1af180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T04:31:59.549 INFO:teuthology.orchestra.run.smithi178.stdout:9 2024-04-26T04:31:59.565 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 8c4f8df5-465a-486e-9413-1de14e33c119 10 2024-04-26T04:31:59.585 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:59.580+0000 7f45781a1180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T04:31:59.708 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:59.704+0000 7f2f6b1af180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T04:31:59.740 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:59.736+0000 7f45781a1180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T04:31:59.765 INFO:teuthology.orchestra.run.smithi178.stdout:10 2024-04-26T04:31:59.774 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:59.772+0000 7f2f6b1af180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T04:31:59.782 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 6fe54c2d-0b95-4235-8c22-51230cfaa591 11 2024-04-26T04:31:59.805 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:59.800+0000 7f45781a1180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T04:31:59.915 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:31:59.912+0000 7f2f6b1af180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T04:31:59.943 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:31:59.940+0000 7f45781a1180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T04:31:59.996 INFO:teuthology.orchestra.run.smithi178.stdout:11 2024-04-26T04:32:00.012 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 5ddd0f05-fb78-43bb-95ec-3fc7985603b3 12 2024-04-26T04:32:00.202 INFO:teuthology.orchestra.run.smithi178.stdout:12 2024-04-26T04:32:00.218 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 37eb42c1-f29e-49f5-b8f6-035178038e5d 13 2024-04-26T04:32:00.420 INFO:teuthology.orchestra.run.smithi178.stdout:13 2024-04-26T04:32:00.437 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new b2267367-a857-4a5c-a8d1-6a9b23c748f6 14 2024-04-26T04:32:00.566 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:32:00.560+0000 7f2f6b1af180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T04:32:00.581 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:32:00.576+0000 7f45781a1180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T04:32:00.628 INFO:teuthology.orchestra.run.smithi178.stdout:14 2024-04-26T04:32:00.644 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 9933b5aa-e091-43b1-b788-5b79695bab11 15 2024-04-26T04:32:00.646 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:32:00.640+0000 7f2f6b1af180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T04:32:00.662 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:32:00.656+0000 7f45781a1180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T04:32:00.833 INFO:teuthology.orchestra.run.smithi178.stdout:15 2024-04-26T04:32:00.836 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:32:00.832+0000 7f2f6b1af180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T04:32:00.845 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:32:00.840+0000 7f45781a1180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T04:32:00.850 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T04:32:00.850 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T04:32:00.852 INFO:tasks.ceph.osd.0:Started 2024-04-26T04:32:00.852 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T04:32:00.852 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T04:32:00.855 INFO:tasks.ceph.osd.4:Started 2024-04-26T04:32:00.855 INFO:tasks.ceph.osd.8:Restarting daemon 2024-04-26T04:32:00.855 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-04-26T04:32:00.857 INFO:tasks.ceph.osd.8:Started 2024-04-26T04:32:00.857 INFO:tasks.ceph.osd.12:Restarting daemon 2024-04-26T04:32:00.858 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 12 2024-04-26T04:32:00.860 INFO:tasks.ceph.osd.12:Started 2024-04-26T04:32:00.860 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T04:32:00.860 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T04:32:00.862 INFO:tasks.ceph.osd.1:Started 2024-04-26T04:32:00.862 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T04:32:00.862 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T04:32:00.865 INFO:tasks.ceph.osd.5:Started 2024-04-26T04:32:00.865 INFO:tasks.ceph.osd.9:Restarting daemon 2024-04-26T04:32:00.865 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2024-04-26T04:32:00.867 INFO:tasks.ceph.osd.9:Started 2024-04-26T04:32:00.867 INFO:tasks.ceph.osd.13:Restarting daemon 2024-04-26T04:32:00.868 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 13 2024-04-26T04:32:00.870 INFO:tasks.ceph.osd.13:Started 2024-04-26T04:32:00.870 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T04:32:00.870 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T04:32:00.873 INFO:tasks.ceph.osd.2:Started 2024-04-26T04:32:00.873 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T04:32:00.873 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T04:32:00.875 INFO:tasks.ceph.osd.6:Started 2024-04-26T04:32:00.876 INFO:tasks.ceph.osd.10:Restarting daemon 2024-04-26T04:32:00.876 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 10 2024-04-26T04:32:00.878 INFO:tasks.ceph.osd.10:Started 2024-04-26T04:32:00.878 INFO:tasks.ceph.osd.14:Restarting daemon 2024-04-26T04:32:00.878 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 14 2024-04-26T04:32:00.881 INFO:tasks.ceph.osd.14:Started 2024-04-26T04:32:00.881 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T04:32:00.881 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T04:32:00.895 INFO:tasks.ceph.osd.3:Started 2024-04-26T04:32:00.895 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T04:32:00.895 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T04:32:00.897 INFO:tasks.ceph.osd.7:Started 2024-04-26T04:32:00.897 INFO:tasks.ceph.osd.11:Restarting daemon 2024-04-26T04:32:00.898 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 11 2024-04-26T04:32:00.900 INFO:tasks.ceph.osd.11:Started 2024-04-26T04:32:00.900 INFO:tasks.ceph.osd.15:Restarting daemon 2024-04-26T04:32:00.900 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 15 2024-04-26T04:32:00.902 INFO:tasks.ceph.osd.15:Started 2024-04-26T04:32:00.903 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T04:32:00.932 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:32:00.928+0000 7f2f6b1af180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T04:32:00.932 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:32:00.928+0000 7f45781a1180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T04:32:01.084 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:01.084 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":18,"fsid":"12c27fbe-0161-48d0-88ec-42a226685437","created":"2024-04-26T04:31:56.451062+0000","modified":"2024-04-26T04:32:00.829165+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T04:32:00.829165+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"55a4f254-2c2a-4007-9c52-b89df6fffb69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"32db4f97-c8ac-481c-9e80-18fc91004298","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2cd26466-5d59-4599-ba3f-a78b0e3b681f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0945017d-9525-49d6-aa67-6635acd94c3e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"38dc68bf-0a89-4900-9d35-24c70368447a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e91cc8a1-57e1-4760-a329-8a60aa752a95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a1322130-3391-432d-a1ee-f83297885029","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"02ee4ca2-9d48-4aff-9641-1823760c7a3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"ea3d5f62-3d68-4824-8556-da8db36553fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8c4f8df5-465a-486e-9413-1de14e33c119","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"6fe54c2d-0b95-4235-8c22-51230cfaa591","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"5ddd0f05-fb78-43bb-95ec-3fc7985603b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"37eb42c1-f29e-49f5-b8f6-035178038e5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"b2267367-a857-4a5c-a8d1-6a9b23c748f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"9933b5aa-e091-43b1-b788-5b79695bab11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:32:01.100 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T04:32:01.100 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T04:32:01.229 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:32:01.224+0000 7f45781a1180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T04:32:01.244 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:32:01.240+0000 7f2f6b1af180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T04:32:01.290 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:32:01.288+0000 7f45781a1180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T04:32:01.309 INFO:tasks.ceph.mgr.y.smithi012.stderr:2024-04-26T04:32:01.304+0000 7f2f6b1af180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T04:32:01.468 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:01.464+0000 7f28885367c0 -1 Falling back to public interface 2024-04-26T04:32:01.479 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:01.476+0000 7fc5aff6f7c0 -1 Falling back to public interface 2024-04-26T04:32:01.487 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:01.484+0000 7f207a2fb7c0 -1 Falling back to public interface 2024-04-26T04:32:01.491 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:01.488+0000 7f93eafac7c0 -1 Falling back to public interface 2024-04-26T04:32:01.505 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:01.500+0000 7fd349e447c0 -1 Falling back to public interface 2024-04-26T04:32:01.505 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:01.500+0000 7f806a0457c0 -1 Falling back to public interface 2024-04-26T04:32:01.505 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:01.500+0000 7fd348e04640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6805/0: (98) Address already in use 2024-04-26T04:32:01.509 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:01.504+0000 7f424cbfd7c0 -1 Falling back to public interface 2024-04-26T04:32:01.509 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:01.504+0000 7fe455cd07c0 -1 Falling back to public interface 2024-04-26T04:32:01.514 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:01.512+0000 7f76098827c0 -1 Falling back to public interface 2024-04-26T04:32:01.518 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:01.516+0000 7fc7fb97a7c0 -1 Falling back to public interface 2024-04-26T04:32:01.523 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:01.520+0000 7fc452b417c0 -1 Falling back to public interface 2024-04-26T04:32:01.527 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:01.524+0000 7f377112a7c0 -1 Falling back to public interface 2024-04-26T04:32:01.548 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:01.548+0000 7fac9f6907c0 -1 Falling back to public interface 2024-04-26T04:32:01.555 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:01.556+0000 7f66826d57c0 -1 Falling back to public interface 2024-04-26T04:32:01.559 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:01.560+0000 7ffac445e7c0 -1 Falling back to public interface 2024-04-26T04:32:01.567 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:01.568+0000 7ff1e8c0d7c0 -1 Falling back to public interface 2024-04-26T04:32:02.339 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:32:02.336+0000 7f451f6e9640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T04:32:02.340 INFO:tasks.ceph.mgr.x.smithi178.stderr:2024-04-26T04:32:02.336+0000 7f451f6e9640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T04:32:02.708 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T04:32:03.024 INFO:teuthology.misc.health.smithi012.stdout: 2024-04-26T04:32:03.024 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":18,"fsid":"12c27fbe-0161-48d0-88ec-42a226685437","created":"2024-04-26T04:31:56.451062+0000","modified":"2024-04-26T04:32:00.829165+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T04:32:00.829165+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"55a4f254-2c2a-4007-9c52-b89df6fffb69","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"32db4f97-c8ac-481c-9e80-18fc91004298","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2cd26466-5d59-4599-ba3f-a78b0e3b681f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0945017d-9525-49d6-aa67-6635acd94c3e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"38dc68bf-0a89-4900-9d35-24c70368447a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e91cc8a1-57e1-4760-a329-8a60aa752a95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a1322130-3391-432d-a1ee-f83297885029","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"02ee4ca2-9d48-4aff-9641-1823760c7a3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"ea3d5f62-3d68-4824-8556-da8db36553fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"8c4f8df5-465a-486e-9413-1de14e33c119","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"6fe54c2d-0b95-4235-8c22-51230cfaa591","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"5ddd0f05-fb78-43bb-95ec-3fc7985603b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"37eb42c1-f29e-49f5-b8f6-035178038e5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"b2267367-a857-4a5c-a8d1-6a9b23c748f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"9933b5aa-e091-43b1-b788-5b79695bab11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:32:03.041 DEBUG:teuthology.misc:0 of 16 OSDs are up 2024-04-26T04:32:05.196 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:05.192+0000 7f28885367c0 -1 osd.8 0 log_to_monitors true 2024-04-26T04:32:05.224 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:05.220+0000 7fe455cd07c0 -1 osd.10 0 log_to_monitors true 2024-04-26T04:32:05.233 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:05.228+0000 7fd349e447c0 -1 osd.15 0 log_to_monitors true 2024-04-26T04:32:05.301 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:05.296+0000 7f207a2fb7c0 -1 osd.0 0 log_to_monitors true 2024-04-26T04:32:05.314 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:05.308+0000 7fc5aff6f7c0 -1 osd.4 0 log_to_monitors true 2024-04-26T04:32:05.326 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:05.328+0000 7fac9f6907c0 -1 osd.5 0 log_to_monitors true 2024-04-26T04:32:05.327 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:05.324+0000 7f76098827c0 -1 osd.11 0 log_to_monitors true 2024-04-26T04:32:05.330 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:05.324+0000 7f377112a7c0 -1 osd.14 0 log_to_monitors true 2024-04-26T04:32:05.362 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:05.360+0000 7fc452b417c0 -1 osd.6 0 log_to_monitors true 2024-04-26T04:32:05.387 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:05.384+0000 7f424cbfd7c0 -1 osd.2 0 log_to_monitors true 2024-04-26T04:32:05.402 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:05.400+0000 7f93eafac7c0 -1 osd.12 0 log_to_monitors true 2024-04-26T04:32:05.417 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:05.416+0000 7f66826d57c0 -1 osd.1 0 log_to_monitors true 2024-04-26T04:32:05.426 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:05.424+0000 7f806a0457c0 -1 osd.3 0 log_to_monitors true 2024-04-26T04:32:05.428 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:05.424+0000 7fc7fb97a7c0 -1 osd.7 0 log_to_monitors true 2024-04-26T04:32:05.434 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:05.432+0000 7ffac445e7c0 -1 osd.13 0 log_to_monitors true 2024-04-26T04:32:05.549 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:05.548+0000 7ff1e8c0d7c0 -1 osd.9 0 log_to_monitors true 2024-04-26T04:32:06.438 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:06.436+0000 7f2883cbc640 -1 osd.8 0 waiting for initial osdmap 2024-04-26T04:32:06.439 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:06.436+0000 7fc5acf2b640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T04:32:06.440 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:06.436+0000 7fd346e00640 -1 osd.15 0 waiting for initial osdmap 2024-04-26T04:32:06.446 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:06.444+0000 7fe451456640 -1 osd.10 0 waiting for initial osdmap 2024-04-26T04:32:06.456 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:06.452+0000 7fd341c19640 -1 osd.15 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:06.464 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:06.460+0000 7fe44daa5640 -1 osd.10 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:06.465 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:06.460+0000 7f288030b640 -1 osd.8 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:06.466 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:06.464+0000 7fc5a7d44640 -1 osd.4 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:07.207 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:07.204+0000 7f20772b7640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T04:32:07.235 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:07.232+0000 7f206e0c8640 -1 osd.0 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:07.373 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:07.368+0000 7f80657cb640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T04:32:07.399 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:07.396+0000 7f8061e1a640 -1 osd.3 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:08.301 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:08.296+0000 7f7605008640 -1 osd.11 0 waiting for initial osdmap 2024-04-26T04:32:08.323 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:08.320+0000 7f75fd64f640 -1 osd.11 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:08.335 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:08.332+0000 7fc7f7100640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T04:32:08.363 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:08.360+0000 7fc7ef747640 -1 osd.7 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:08.399 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:08.396+0000 7f93e6732640 -1 osd.12 0 waiting for initial osdmap 2024-04-26T04:32:08.443 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:08.440+0000 7f93ded79640 -1 osd.12 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:09.003 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:09.004+0000 7ffac141a640 -1 osd.13 0 waiting for initial osdmap 2024-04-26T04:32:09.006 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:09.004+0000 7fac9c64c640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T04:32:09.036 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:09.036+0000 7ffab822b640 -1 osd.13 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:09.040 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:09.040+0000 7ff1e4393640 -1 osd.9 0 waiting for initial osdmap 2024-04-26T04:32:09.050 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:09.048+0000 7fac9345d640 -1 osd.5 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:09.067 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:09.064+0000 7f667de5b640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T04:32:09.104 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:09.104+0000 7f66764a2640 -1 osd.1 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:09.121 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:09.116+0000 7fc44e2c7640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T04:32:09.168 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:09.164+0000 7f376c8b0640 -1 osd.14 0 waiting for initial osdmap 2024-04-26T04:32:09.172 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:09.172+0000 7ff1dc9da640 -1 osd.9 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:09.177 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:09.172+0000 7f4248383640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T04:32:09.260 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:09.256+0000 7fc44690e640 -1 osd.6 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:09.300 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:09.296+0000 7f3764ef7640 -1 osd.14 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:09.315 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:09.312+0000 7f42409ca640 -1 osd.2 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:32:10.652 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T04:32:10.961 INFO:teuthology.misc.health.smithi012.stdout: 2024-04-26T04:32:10.961 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":24,"fsid":"12c27fbe-0161-48d0-88ec-42a226685437","created":"2024-04-26T04:31:56.451062+0000","modified":"2024-04-26T04:32:10.455853+0000","last_up_change":"2024-04-26T04:32:09.452464+0000","last_in_change":"2024-04-26T04:32:00.829165+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:32:07.426875+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"55a4f254-2c2a-4007-9c52-b89df6fffb69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6817","nonce":1754063127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6819","nonce":1754063127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6823","nonce":1754063127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6821","nonce":1754063127}]},"public_addr":"172.21.15.12:6817/1754063127","cluster_addr":"172.21.15.12:6819/1754063127","heartbeat_back_addr":"172.21.15.12:6823/1754063127","heartbeat_front_addr":"172.21.15.12:6821/1754063127","state":["exists","up"]},{"osd":1,"uuid":"32db4f97-c8ac-481c-9e80-18fc91004298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6809","nonce":3264326676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6811","nonce":3264326676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6817","nonce":3264326676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6813","nonce":3264326676}]},"public_addr":"172.21.15.113:6809/3264326676","cluster_addr":"172.21.15.113:6811/3264326676","heartbeat_back_addr":"172.21.15.113:6817/3264326676","heartbeat_front_addr":"172.21.15.113:6813/3264326676","state":["exists","up"]},{"osd":2,"uuid":"2cd26466-5d59-4599-ba3f-a78b0e3b681f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6802","nonce":1454475448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6806","nonce":1454475448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6814","nonce":1454475448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6810","nonce":1454475448}]},"public_addr":"172.21.15.137:6802/1454475448","cluster_addr":"172.21.15.137:6806/1454475448","heartbeat_back_addr":"172.21.15.137:6814/1454475448","heartbeat_front_addr":"172.21.15.137:6810/1454475448","state":["exists","up"]},{"osd":3,"uuid":"0945017d-9525-49d6-aa67-6635acd94c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6803","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6805","nonce":2898698985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6810","nonce":2898698985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6817","nonce":2898698985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6815","nonce":2898698985}]},"public_addr":"172.21.15.178:6805/2898698985","cluster_addr":"172.21.15.178:6810/2898698985","heartbeat_back_addr":"172.21.15.178:6817/2898698985","heartbeat_front_addr":"172.21.15.178:6815/2898698985","state":["exists","up"]},{"osd":4,"uuid":"38dc68bf-0a89-4900-9d35-24c70368447a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6809","nonce":3209840823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6811","nonce":3209840823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6815","nonce":3209840823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6813","nonce":3209840823}]},"public_addr":"172.21.15.12:6809/3209840823","cluster_addr":"172.21.15.12:6811/3209840823","heartbeat_back_addr":"172.21.15.12:6815/3209840823","heartbeat_front_addr":"172.21.15.12:6813/3209840823","state":["exists","up"]},{"osd":5,"uuid":"e91cc8a1-57e1-4760-a329-8a60aa752a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6801","nonce":1475637773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6803","nonce":1475637773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6807","nonce":1475637773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6805","nonce":1475637773}]},"public_addr":"172.21.15.113:6801/1475637773","cluster_addr":"172.21.15.113:6803/1475637773","heartbeat_back_addr":"172.21.15.113:6807/1475637773","heartbeat_front_addr":"172.21.15.113:6805/1475637773","state":["exists","up"]},{"osd":6,"uuid":"a1322130-3391-432d-a1ee-f83297885029","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6817","nonce":771338783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6819","nonce":771338783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6823","nonce":771338783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6821","nonce":771338783}]},"public_addr":"172.21.15.137:6817/771338783","cluster_addr":"172.21.15.137:6819/771338783","heartbeat_back_addr":"172.21.15.137:6823/771338783","heartbeat_front_addr":"172.21.15.137:6821/771338783","state":["exists","up"]},{"osd":7,"uuid":"02ee4ca2-9d48-4aff-9641-1823760c7a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6827","nonce":1017152077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6829","nonce":1017152077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6833","nonce":1017152077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6831","nonce":1017152077}]},"public_addr":"172.21.15.178:6827/1017152077","cluster_addr":"172.21.15.178:6829/1017152077","heartbeat_back_addr":"172.21.15.178:6833/1017152077","heartbeat_front_addr":"172.21.15.178:6831/1017152077","state":["exists","up"]},{"osd":8,"uuid":"ea3d5f62-3d68-4824-8556-da8db36553fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6801","nonce":2153129138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6803","nonce":2153129138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6807","nonce":2153129138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6805","nonce":2153129138}]},"public_addr":"172.21.15.12:6801/2153129138","cluster_addr":"172.21.15.12:6803/2153129138","heartbeat_back_addr":"172.21.15.12:6807/2153129138","heartbeat_front_addr":"172.21.15.12:6805/2153129138","state":["exists","up"]},{"osd":9,"uuid":"e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6825","nonce":359931551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6827","nonce":359931551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6831","nonce":359931551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6829","nonce":359931551}]},"public_addr":"172.21.15.113:6825/359931551","cluster_addr":"172.21.15.113:6827/359931551","heartbeat_back_addr":"172.21.15.113:6831/359931551","heartbeat_front_addr":"172.21.15.113:6829/359931551","state":["exists","up"]},{"osd":10,"uuid":"8c4f8df5-465a-486e-9413-1de14e33c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6801","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6803","nonce":1533480861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6807","nonce":1533480861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6815","nonce":1533480861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6811","nonce":1533480861}]},"public_addr":"172.21.15.137:6803/1533480861","cluster_addr":"172.21.15.137:6807/1533480861","heartbeat_back_addr":"172.21.15.137:6815/1533480861","heartbeat_front_addr":"172.21.15.137:6811/1533480861","state":["exists","up"]},{"osd":11,"uuid":"6fe54c2d-0b95-4235-8c22-51230cfaa591","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6819","nonce":1683682957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6821","nonce":1683682957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6825","nonce":1683682957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6823","nonce":1683682957}]},"public_addr":"172.21.15.178:6819/1683682957","cluster_addr":"172.21.15.178:6821/1683682957","heartbeat_back_addr":"172.21.15.178:6825/1683682957","heartbeat_front_addr":"172.21.15.178:6823/1683682957","state":["exists","up"]},{"osd":12,"uuid":"5ddd0f05-fb78-43bb-95ec-3fc7985603b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6825","nonce":2509311840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6827","nonce":2509311840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6831","nonce":2509311840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6829","nonce":2509311840}]},"public_addr":"172.21.15.12:6825/2509311840","cluster_addr":"172.21.15.12:6827/2509311840","heartbeat_back_addr":"172.21.15.12:6831/2509311840","heartbeat_front_addr":"172.21.15.12:6829/2509311840","state":["exists","up"]},{"osd":13,"uuid":"37eb42c1-f29e-49f5-b8f6-035178038e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6816","nonce":3716239663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6819","nonce":3716239663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6823","nonce":3716239663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6821","nonce":3716239663}]},"public_addr":"172.21.15.113:6816/3716239663","cluster_addr":"172.21.15.113:6819/3716239663","heartbeat_back_addr":"172.21.15.113:6823/3716239663","heartbeat_front_addr":"172.21.15.113:6821/3716239663","state":["exists","up"]},{"osd":14,"uuid":"b2267367-a857-4a5c-a8d1-6a9b23c748f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6825","nonce":1454956132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6827","nonce":1454956132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6831","nonce":1454956132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6829","nonce":1454956132}]},"public_addr":"172.21.15.137:6825/1454956132","cluster_addr":"172.21.15.137:6827/1454956132","heartbeat_back_addr":"172.21.15.137:6831/1454956132","heartbeat_front_addr":"172.21.15.137:6829/1454956132","state":["exists","up"]},{"osd":15,"uuid":"9933b5aa-e091-43b1-b788-5b79695bab11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6804","nonce":314165740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6807","nonce":314165740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6814","nonce":314165740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6809","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6811","nonce":314165740}]},"public_addr":"172.21.15.178:6804/314165740","cluster_addr":"172.21.15.178:6807/314165740","heartbeat_back_addr":"172.21.15.178:6814/314165740","heartbeat_front_addr":"172.21.15.178:6811/314165740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.305255+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.420004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.396744+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.471304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.276649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.349822+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.396513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.438157+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.166681+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.532725+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.242376+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.328959+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.440612+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.485079+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.290530+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.232637+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:32:10.980 DEBUG:teuthology.misc:16 of 16 OSDs are up 2024-04-26T04:32:10.980 INFO:tasks.ceph:Creating RBD pool 2024-04-26T04:32:10.980 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-26T04:32:11.481 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'rbd' created 2024-04-26T04:32:11.502 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-26T04:32:12.490 INFO:teuthology.orchestra.run.smithi012.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-26T04:32:12.515 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T04:32:12.515 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T04:32:12.515 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T04:32:12.515 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T04:32:12.516 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T04:32:12.868 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:12.885 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":4,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":3657255575},{"type":"v1","addr":"172.21.15.178:6801","nonce":3657255575}]},"active_addr":"172.21.15.178:6801/3657255575","active_change":"2024-04-26T04:32:01.311600+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":2693986622}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":446887574}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":3372061761}]}]} 2024-04-26T04:32:12.889 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T04:32:12.889 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T04:32:12.889 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T04:32:13.196 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:13.196 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":26,"fsid":"12c27fbe-0161-48d0-88ec-42a226685437","created":"2024-04-26T04:31:56.451062+0000","modified":"2024-04-26T04:32:12.477831+0000","last_up_change":"2024-04-26T04:32:09.452464+0000","last_in_change":"2024-04-26T04:32:00.829165+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:32:07.426875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-26T04:32:11.318957+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"55a4f254-2c2a-4007-9c52-b89df6fffb69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6817","nonce":1754063127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6819","nonce":1754063127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6823","nonce":1754063127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6821","nonce":1754063127}]},"public_addr":"172.21.15.12:6817/1754063127","cluster_addr":"172.21.15.12:6819/1754063127","heartbeat_back_addr":"172.21.15.12:6823/1754063127","heartbeat_front_addr":"172.21.15.12:6821/1754063127","state":["exists","up"]},{"osd":1,"uuid":"32db4f97-c8ac-481c-9e80-18fc91004298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6809","nonce":3264326676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6811","nonce":3264326676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6817","nonce":3264326676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6813","nonce":3264326676}]},"public_addr":"172.21.15.113:6809/3264326676","cluster_addr":"172.21.15.113:6811/3264326676","heartbeat_back_addr":"172.21.15.113:6817/3264326676","heartbeat_front_addr":"172.21.15.113:6813/3264326676","state":["exists","up"]},{"osd":2,"uuid":"2cd26466-5d59-4599-ba3f-a78b0e3b681f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6802","nonce":1454475448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6806","nonce":1454475448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6814","nonce":1454475448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6810","nonce":1454475448}]},"public_addr":"172.21.15.137:6802/1454475448","cluster_addr":"172.21.15.137:6806/1454475448","heartbeat_back_addr":"172.21.15.137:6814/1454475448","heartbeat_front_addr":"172.21.15.137:6810/1454475448","state":["exists","up"]},{"osd":3,"uuid":"0945017d-9525-49d6-aa67-6635acd94c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6803","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6805","nonce":2898698985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6810","nonce":2898698985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6817","nonce":2898698985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6815","nonce":2898698985}]},"public_addr":"172.21.15.178:6805/2898698985","cluster_addr":"172.21.15.178:6810/2898698985","heartbeat_back_addr":"172.21.15.178:6817/2898698985","heartbeat_front_addr":"172.21.15.178:6815/2898698985","state":["exists","up"]},{"osd":4,"uuid":"38dc68bf-0a89-4900-9d35-24c70368447a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6809","nonce":3209840823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6811","nonce":3209840823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6815","nonce":3209840823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6813","nonce":3209840823}]},"public_addr":"172.21.15.12:6809/3209840823","cluster_addr":"172.21.15.12:6811/3209840823","heartbeat_back_addr":"172.21.15.12:6815/3209840823","heartbeat_front_addr":"172.21.15.12:6813/3209840823","state":["exists","up"]},{"osd":5,"uuid":"e91cc8a1-57e1-4760-a329-8a60aa752a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6801","nonce":1475637773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6803","nonce":1475637773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6807","nonce":1475637773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6805","nonce":1475637773}]},"public_addr":"172.21.15.113:6801/1475637773","cluster_addr":"172.21.15.113:6803/1475637773","heartbeat_back_addr":"172.21.15.113:6807/1475637773","heartbeat_front_addr":"172.21.15.113:6805/1475637773","state":["exists","up"]},{"osd":6,"uuid":"a1322130-3391-432d-a1ee-f83297885029","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6817","nonce":771338783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6819","nonce":771338783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6823","nonce":771338783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6821","nonce":771338783}]},"public_addr":"172.21.15.137:6817/771338783","cluster_addr":"172.21.15.137:6819/771338783","heartbeat_back_addr":"172.21.15.137:6823/771338783","heartbeat_front_addr":"172.21.15.137:6821/771338783","state":["exists","up"]},{"osd":7,"uuid":"02ee4ca2-9d48-4aff-9641-1823760c7a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6827","nonce":1017152077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6829","nonce":1017152077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6833","nonce":1017152077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6831","nonce":1017152077}]},"public_addr":"172.21.15.178:6827/1017152077","cluster_addr":"172.21.15.178:6829/1017152077","heartbeat_back_addr":"172.21.15.178:6833/1017152077","heartbeat_front_addr":"172.21.15.178:6831/1017152077","state":["exists","up"]},{"osd":8,"uuid":"ea3d5f62-3d68-4824-8556-da8db36553fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6801","nonce":2153129138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6803","nonce":2153129138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6807","nonce":2153129138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6805","nonce":2153129138}]},"public_addr":"172.21.15.12:6801/2153129138","cluster_addr":"172.21.15.12:6803/2153129138","heartbeat_back_addr":"172.21.15.12:6807/2153129138","heartbeat_front_addr":"172.21.15.12:6805/2153129138","state":["exists","up"]},{"osd":9,"uuid":"e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6825","nonce":359931551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6827","nonce":359931551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6831","nonce":359931551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6829","nonce":359931551}]},"public_addr":"172.21.15.113:6825/359931551","cluster_addr":"172.21.15.113:6827/359931551","heartbeat_back_addr":"172.21.15.113:6831/359931551","heartbeat_front_addr":"172.21.15.113:6829/359931551","state":["exists","up"]},{"osd":10,"uuid":"8c4f8df5-465a-486e-9413-1de14e33c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6801","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6803","nonce":1533480861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6807","nonce":1533480861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6815","nonce":1533480861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6811","nonce":1533480861}]},"public_addr":"172.21.15.137:6803/1533480861","cluster_addr":"172.21.15.137:6807/1533480861","heartbeat_back_addr":"172.21.15.137:6815/1533480861","heartbeat_front_addr":"172.21.15.137:6811/1533480861","state":["exists","up"]},{"osd":11,"uuid":"6fe54c2d-0b95-4235-8c22-51230cfaa591","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6819","nonce":1683682957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6821","nonce":1683682957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6825","nonce":1683682957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6823","nonce":1683682957}]},"public_addr":"172.21.15.178:6819/1683682957","cluster_addr":"172.21.15.178:6821/1683682957","heartbeat_back_addr":"172.21.15.178:6825/1683682957","heartbeat_front_addr":"172.21.15.178:6823/1683682957","state":["exists","up"]},{"osd":12,"uuid":"5ddd0f05-fb78-43bb-95ec-3fc7985603b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6825","nonce":2509311840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6827","nonce":2509311840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6831","nonce":2509311840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6829","nonce":2509311840}]},"public_addr":"172.21.15.12:6825/2509311840","cluster_addr":"172.21.15.12:6827/2509311840","heartbeat_back_addr":"172.21.15.12:6831/2509311840","heartbeat_front_addr":"172.21.15.12:6829/2509311840","state":["exists","up"]},{"osd":13,"uuid":"37eb42c1-f29e-49f5-b8f6-035178038e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6816","nonce":3716239663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6819","nonce":3716239663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6823","nonce":3716239663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6821","nonce":3716239663}]},"public_addr":"172.21.15.113:6816/3716239663","cluster_addr":"172.21.15.113:6819/3716239663","heartbeat_back_addr":"172.21.15.113:6823/3716239663","heartbeat_front_addr":"172.21.15.113:6821/3716239663","state":["exists","up"]},{"osd":14,"uuid":"b2267367-a857-4a5c-a8d1-6a9b23c748f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6825","nonce":1454956132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6827","nonce":1454956132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6831","nonce":1454956132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6829","nonce":1454956132}]},"public_addr":"172.21.15.137:6825/1454956132","cluster_addr":"172.21.15.137:6827/1454956132","heartbeat_back_addr":"172.21.15.137:6831/1454956132","heartbeat_front_addr":"172.21.15.137:6829/1454956132","state":["exists","up"]},{"osd":15,"uuid":"9933b5aa-e091-43b1-b788-5b79695bab11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6804","nonce":314165740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6807","nonce":314165740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6814","nonce":314165740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6809","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6811","nonce":314165740}]},"public_addr":"172.21.15.178:6804/314165740","cluster_addr":"172.21.15.178:6807/314165740","heartbeat_back_addr":"172.21.15.178:6814/314165740","heartbeat_front_addr":"172.21.15.178:6811/314165740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.305255+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.420004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.396744+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.471304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.276649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.349822+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.396513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.438157+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.166681+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.532725+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.242376+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.328959+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.440612+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.485079+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.290530+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.232637+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:32:13.214 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T04:32:13.215 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T04:32:13.510 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:13.511 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":27,"fsid":"12c27fbe-0161-48d0-88ec-42a226685437","created":"2024-04-26T04:31:56.451062+0000","modified":"2024-04-26T04:32:13.489645+0000","last_up_change":"2024-04-26T04:32:09.452464+0000","last_in_change":"2024-04-26T04:32:00.829165+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:32:07.426875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-26T04:32:11.318957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"55a4f254-2c2a-4007-9c52-b89df6fffb69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6817","nonce":1754063127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6819","nonce":1754063127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6823","nonce":1754063127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6821","nonce":1754063127}]},"public_addr":"172.21.15.12:6817/1754063127","cluster_addr":"172.21.15.12:6819/1754063127","heartbeat_back_addr":"172.21.15.12:6823/1754063127","heartbeat_front_addr":"172.21.15.12:6821/1754063127","state":["exists","up"]},{"osd":1,"uuid":"32db4f97-c8ac-481c-9e80-18fc91004298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6809","nonce":3264326676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6811","nonce":3264326676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6817","nonce":3264326676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6813","nonce":3264326676}]},"public_addr":"172.21.15.113:6809/3264326676","cluster_addr":"172.21.15.113:6811/3264326676","heartbeat_back_addr":"172.21.15.113:6817/3264326676","heartbeat_front_addr":"172.21.15.113:6813/3264326676","state":["exists","up"]},{"osd":2,"uuid":"2cd26466-5d59-4599-ba3f-a78b0e3b681f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6802","nonce":1454475448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6806","nonce":1454475448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6814","nonce":1454475448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6810","nonce":1454475448}]},"public_addr":"172.21.15.137:6802/1454475448","cluster_addr":"172.21.15.137:6806/1454475448","heartbeat_back_addr":"172.21.15.137:6814/1454475448","heartbeat_front_addr":"172.21.15.137:6810/1454475448","state":["exists","up"]},{"osd":3,"uuid":"0945017d-9525-49d6-aa67-6635acd94c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6803","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6805","nonce":2898698985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6810","nonce":2898698985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6817","nonce":2898698985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6815","nonce":2898698985}]},"public_addr":"172.21.15.178:6805/2898698985","cluster_addr":"172.21.15.178:6810/2898698985","heartbeat_back_addr":"172.21.15.178:6817/2898698985","heartbeat_front_addr":"172.21.15.178:6815/2898698985","state":["exists","up"]},{"osd":4,"uuid":"38dc68bf-0a89-4900-9d35-24c70368447a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6809","nonce":3209840823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6811","nonce":3209840823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6815","nonce":3209840823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6813","nonce":3209840823}]},"public_addr":"172.21.15.12:6809/3209840823","cluster_addr":"172.21.15.12:6811/3209840823","heartbeat_back_addr":"172.21.15.12:6815/3209840823","heartbeat_front_addr":"172.21.15.12:6813/3209840823","state":["exists","up"]},{"osd":5,"uuid":"e91cc8a1-57e1-4760-a329-8a60aa752a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6801","nonce":1475637773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6803","nonce":1475637773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6807","nonce":1475637773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6805","nonce":1475637773}]},"public_addr":"172.21.15.113:6801/1475637773","cluster_addr":"172.21.15.113:6803/1475637773","heartbeat_back_addr":"172.21.15.113:6807/1475637773","heartbeat_front_addr":"172.21.15.113:6805/1475637773","state":["exists","up"]},{"osd":6,"uuid":"a1322130-3391-432d-a1ee-f83297885029","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6817","nonce":771338783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6819","nonce":771338783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6823","nonce":771338783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6821","nonce":771338783}]},"public_addr":"172.21.15.137:6817/771338783","cluster_addr":"172.21.15.137:6819/771338783","heartbeat_back_addr":"172.21.15.137:6823/771338783","heartbeat_front_addr":"172.21.15.137:6821/771338783","state":["exists","up"]},{"osd":7,"uuid":"02ee4ca2-9d48-4aff-9641-1823760c7a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6827","nonce":1017152077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6829","nonce":1017152077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6833","nonce":1017152077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6831","nonce":1017152077}]},"public_addr":"172.21.15.178:6827/1017152077","cluster_addr":"172.21.15.178:6829/1017152077","heartbeat_back_addr":"172.21.15.178:6833/1017152077","heartbeat_front_addr":"172.21.15.178:6831/1017152077","state":["exists","up"]},{"osd":8,"uuid":"ea3d5f62-3d68-4824-8556-da8db36553fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6801","nonce":2153129138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6803","nonce":2153129138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6807","nonce":2153129138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6805","nonce":2153129138}]},"public_addr":"172.21.15.12:6801/2153129138","cluster_addr":"172.21.15.12:6803/2153129138","heartbeat_back_addr":"172.21.15.12:6807/2153129138","heartbeat_front_addr":"172.21.15.12:6805/2153129138","state":["exists","up"]},{"osd":9,"uuid":"e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6825","nonce":359931551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6827","nonce":359931551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6831","nonce":359931551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6829","nonce":359931551}]},"public_addr":"172.21.15.113:6825/359931551","cluster_addr":"172.21.15.113:6827/359931551","heartbeat_back_addr":"172.21.15.113:6831/359931551","heartbeat_front_addr":"172.21.15.113:6829/359931551","state":["exists","up"]},{"osd":10,"uuid":"8c4f8df5-465a-486e-9413-1de14e33c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6801","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6803","nonce":1533480861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6807","nonce":1533480861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6815","nonce":1533480861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6811","nonce":1533480861}]},"public_addr":"172.21.15.137:6803/1533480861","cluster_addr":"172.21.15.137:6807/1533480861","heartbeat_back_addr":"172.21.15.137:6815/1533480861","heartbeat_front_addr":"172.21.15.137:6811/1533480861","state":["exists","up"]},{"osd":11,"uuid":"6fe54c2d-0b95-4235-8c22-51230cfaa591","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6819","nonce":1683682957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6821","nonce":1683682957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6825","nonce":1683682957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6823","nonce":1683682957}]},"public_addr":"172.21.15.178:6819/1683682957","cluster_addr":"172.21.15.178:6821/1683682957","heartbeat_back_addr":"172.21.15.178:6825/1683682957","heartbeat_front_addr":"172.21.15.178:6823/1683682957","state":["exists","up"]},{"osd":12,"uuid":"5ddd0f05-fb78-43bb-95ec-3fc7985603b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6825","nonce":2509311840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6827","nonce":2509311840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6831","nonce":2509311840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6829","nonce":2509311840}]},"public_addr":"172.21.15.12:6825/2509311840","cluster_addr":"172.21.15.12:6827/2509311840","heartbeat_back_addr":"172.21.15.12:6831/2509311840","heartbeat_front_addr":"172.21.15.12:6829/2509311840","state":["exists","up"]},{"osd":13,"uuid":"37eb42c1-f29e-49f5-b8f6-035178038e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6816","nonce":3716239663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6819","nonce":3716239663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6823","nonce":3716239663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6821","nonce":3716239663}]},"public_addr":"172.21.15.113:6816/3716239663","cluster_addr":"172.21.15.113:6819/3716239663","heartbeat_back_addr":"172.21.15.113:6823/3716239663","heartbeat_front_addr":"172.21.15.113:6821/3716239663","state":["exists","up"]},{"osd":14,"uuid":"b2267367-a857-4a5c-a8d1-6a9b23c748f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6825","nonce":1454956132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6827","nonce":1454956132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6831","nonce":1454956132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6829","nonce":1454956132}]},"public_addr":"172.21.15.137:6825/1454956132","cluster_addr":"172.21.15.137:6827/1454956132","heartbeat_back_addr":"172.21.15.137:6831/1454956132","heartbeat_front_addr":"172.21.15.137:6829/1454956132","state":["exists","up"]},{"osd":15,"uuid":"9933b5aa-e091-43b1-b788-5b79695bab11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6804","nonce":314165740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6807","nonce":314165740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6814","nonce":314165740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6809","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6811","nonce":314165740}]},"public_addr":"172.21.15.178:6804/314165740","cluster_addr":"172.21.15.178:6807/314165740","heartbeat_back_addr":"172.21.15.178:6814/314165740","heartbeat_front_addr":"172.21.15.178:6811/314165740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.305255+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.420004+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.396744+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.471304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.276649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.349822+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.396513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.438157+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.166681+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.532725+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.242376+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.328959+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.440612+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.485079+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.290530+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.232637+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:32:13.529 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T04:32:13.529 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T04:32:13.529 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T04:32:13.530 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T04:32:13.533 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T04:32:13.534 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T04:32:13.534 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T04:32:13.535 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T04:32:13.535 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-04-26T04:32:13.535 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-04-26T04:32:13.536 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-04-26T04:32:13.536 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-04-26T04:32:13.537 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-04-26T04:32:13.537 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-04-26T04:32:13.538 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-04-26T04:32:13.538 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-04-26T04:32:13.901 INFO:teuthology.orchestra.run.smithi012.stdout:90194313220 2024-04-26T04:32:13.901 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-04-26T04:32:13.967 INFO:teuthology.orchestra.run.smithi012.stdout:98784247813 2024-04-26T04:32:13.967 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T04:32:13.980 INFO:teuthology.orchestra.run.smithi012.stdout:98784247811 2024-04-26T04:32:13.980 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T04:32:13.991 INFO:teuthology.orchestra.run.smithi012.stdout:94489280515 2024-04-26T04:32:13.992 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T04:32:14.009 INFO:teuthology.orchestra.run.smithi012.stdout:90194313220 2024-04-26T04:32:14.010 INFO:teuthology.orchestra.run.smithi012.stdout:90194313220 2024-04-26T04:32:14.011 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T04:32:14.011 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T04:32:14.029 INFO:teuthology.orchestra.run.smithi012.stdout:98784247813 2024-04-26T04:32:14.029 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-04-26T04:32:14.066 INFO:teuthology.orchestra.run.smithi012.stdout:98784247812 2024-04-26T04:32:14.066 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-04-26T04:32:14.067 INFO:teuthology.orchestra.run.smithi012.stdout:94489280515 2024-04-26T04:32:14.067 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T04:32:14.074 INFO:teuthology.orchestra.run.smithi012.stdout:98784247813 2024-04-26T04:32:14.075 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-04-26T04:32:14.080 INFO:teuthology.orchestra.run.smithi012.stdout:98784247812 2024-04-26T04:32:14.080 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-04-26T04:32:14.105 INFO:teuthology.orchestra.run.smithi012.stdout:98784247812 2024-04-26T04:32:14.105 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T04:32:14.127 INFO:teuthology.orchestra.run.smithi012.stdout:90194313220 2024-04-26T04:32:14.128 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-04-26T04:32:14.135 INFO:teuthology.orchestra.run.smithi012.stdout:94489280515 2024-04-26T04:32:14.136 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-04-26T04:32:14.142 INFO:teuthology.orchestra.run.smithi012.stdout:98784247813 2024-04-26T04:32:14.143 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T04:32:14.226 INFO:teuthology.orchestra.run.smithi012.stdout:90194313220 2024-04-26T04:32:14.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-04-26T04:32:14.757 INFO:teuthology.orchestra.run.smithi012.stdout:90194313219 2024-04-26T04:32:14.766 INFO:teuthology.orchestra.run.smithi012.stdout:98784247812 2024-04-26T04:32:14.800 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.15 2024-04-26T04:32:14.849 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247812 for osd.2 2024-04-26T04:32:14.881 INFO:teuthology.orchestra.run.smithi012.stdout:94489280514 2024-04-26T04:32:14.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280514 for osd.7 2024-04-26T04:32:14.973 INFO:teuthology.orchestra.run.smithi012.stdout:90194313219 2024-04-26T04:32:15.017 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.4 2024-04-26T04:32:15.023 INFO:teuthology.orchestra.run.smithi012.stdout:90194313219 2024-04-26T04:32:15.053 INFO:teuthology.orchestra.run.smithi012.stdout:98784247810 2024-04-26T04:32:15.082 INFO:teuthology.orchestra.run.smithi012.stdout:94489280514 2024-04-26T04:32:15.095 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.8 2024-04-26T04:32:15.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247810 for osd.6 2024-04-26T04:32:15.109 INFO:teuthology.orchestra.run.smithi012.stdout:98784247812 2024-04-26T04:32:15.109 INFO:teuthology.orchestra.run.smithi012.stdout:94489280514 2024-04-26T04:32:15.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280514 for osd.11 2024-04-26T04:32:15.123 INFO:teuthology.orchestra.run.smithi012.stdout:90194313219 2024-04-26T04:32:15.135 INFO:teuthology.orchestra.run.smithi012.stdout:90194313219 2024-04-26T04:32:15.140 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247812 for osd.5 2024-04-26T04:32:15.151 INFO:teuthology.orchestra.run.smithi012.stdout:98784247812 2024-04-26T04:32:15.152 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280514 for osd.3 2024-04-26T04:32:15.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.0 2024-04-26T04:32:15.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.10 2024-04-26T04:32:15.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247812 for osd.14 2024-04-26T04:32:15.180 INFO:teuthology.orchestra.run.smithi012.stdout:98784247811 2024-04-26T04:32:15.190 INFO:teuthology.orchestra.run.smithi012.stdout:98784247812 2024-04-26T04:32:15.198 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247811 for osd.1 2024-04-26T04:32:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:98784247811 2024-04-26T04:32:15.200 INFO:teuthology.orchestra.run.smithi012.stdout:98784247811 2024-04-26T04:32:15.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247812 for osd.12 2024-04-26T04:32:15.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247811 for osd.9 2024-04-26T04:32:15.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247811 for osd.13 2024-04-26T04:32:15.801 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-04-26T04:32:15.850 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T04:32:15.931 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T04:32:16.019 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T04:32:16.096 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-04-26T04:32:16.096 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T04:32:16.111 INFO:teuthology.orchestra.run.smithi012.stdout:90194313220 2024-04-26T04:32:16.115 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-04-26T04:32:16.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.15 2024-04-26T04:32:16.133 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:16.141 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T04:32:16.153 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T04:32:16.163 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T04:32:16.163 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-04-26T04:32:16.180 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-04-26T04:32:16.190 INFO:teuthology.orchestra.run.smithi012.stdout:98784247813 2024-04-26T04:32:16.199 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T04:32:16.212 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-04-26T04:32:16.222 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-04-26T04:32:16.223 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-04-26T04:32:16.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247813 for osd.2 2024-04-26T04:32:16.226 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:16.425 INFO:teuthology.orchestra.run.smithi012.stdout:94489280516 2024-04-26T04:32:16.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280516 for osd.7 2024-04-26T04:32:16.484 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:16.538 INFO:teuthology.orchestra.run.smithi012.stdout:90194313220 2024-04-26T04:32:16.585 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.4 2024-04-26T04:32:16.585 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:16.825 INFO:teuthology.orchestra.run.smithi012.stdout:98784247811 2024-04-26T04:32:16.827 INFO:teuthology.orchestra.run.smithi012.stdout:94489280516 2024-04-26T04:32:16.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247811 for osd.6 2024-04-26T04:32:16.853 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:16.874 INFO:teuthology.orchestra.run.smithi012.stdout:94489280516 2024-04-26T04:32:16.881 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280516 for osd.3 2024-04-26T04:32:16.881 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:16.917 INFO:teuthology.orchestra.run.smithi012.stdout:90194313220 2024-04-26T04:32:16.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280516 for osd.11 2024-04-26T04:32:16.918 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:16.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.0 2024-04-26T04:32:16.975 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:16.989 INFO:teuthology.orchestra.run.smithi012.stdout:98784247813 2024-04-26T04:32:16.989 INFO:teuthology.orchestra.run.smithi012.stdout:98784247813 2024-04-26T04:32:16.990 INFO:teuthology.orchestra.run.smithi012.stdout:90194313220 2024-04-26T04:32:17.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247813 for osd.14 2024-04-26T04:32:17.019 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:17.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247813 for osd.5 2024-04-26T04:32:17.023 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:17.024 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.8 2024-04-26T04:32:17.024 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:17.053 INFO:teuthology.orchestra.run.smithi012.stdout:90194313220 2024-04-26T04:32:17.061 INFO:teuthology.orchestra.run.smithi012.stdout:98784247812 2024-04-26T04:32:17.070 INFO:teuthology.orchestra.run.smithi012.stdout:98784247812 2024-04-26T04:32:17.079 INFO:teuthology.orchestra.run.smithi012.stdout:98784247813 2024-04-26T04:32:17.081 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.10 2024-04-26T04:32:17.081 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:17.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.13 2024-04-26T04:32:17.082 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:17.090 INFO:teuthology.orchestra.run.smithi012.stdout:98784247812 2024-04-26T04:32:17.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.1 2024-04-26T04:32:17.091 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:17.098 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247813 got 98784247813 for osd.12 2024-04-26T04:32:17.099 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:17.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.9 2024-04-26T04:32:17.108 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:17.108 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T04:32:17.109 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:32:17.426 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:17.426 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:32:17.443 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-04-26T04:32:17.325961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":62,"num_write":92,"num_write_kb":1109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60,"ondisk_log_size":60,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":445388,"kb_used_data":4648,"kb_used_omap":24,"kb_used_meta":440423,"kb_avail":1509504052,"statfs":{"total":1546188226560,"available":1545732149248,"internally_reserved":0,"allocated":4759552,"data_stored":2104870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":450993231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131072,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":28,"num_write_kb":522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000702"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.511264+0000","last_change":"2024-04-26T04:32:12.803651+0000","last_active":"2024-04-26T04:32:13.511264+0000","last_peered":"2024-04-26T04:32:13.511264+0000","last_clean":"2024-04-26T04:32:13.511264+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:32:13.511264+0000","last_undegraded":"2024-04-26T04:32:13.511264+0000","last_fullsized":"2024-04-26T04:32:13.511264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:26.614652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.840131+0000","last_change":"2024-04-26T04:32:12.697838+0000","last_active":"2024-04-26T04:32:13.840131+0000","last_peered":"2024-04-26T04:32:13.840131+0000","last_clean":"2024-04-26T04:32:13.840131+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:32:13.840131+0000","last_undegraded":"2024-04-26T04:32:13.840131+0000","last_fullsized":"2024-04-26T04:32:13.840131+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.961816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.723527+0000","last_change":"2024-04-26T04:32:12.504400+0000","last_active":"2024-04-26T04:32:13.723527+0000","last_peered":"2024-04-26T04:32:13.723527+0000","last_clean":"2024-04-26T04:32:13.723527+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:32:13.723527+0000","last_undegraded":"2024-04-26T04:32:13.723527+0000","last_fullsized":"2024-04-26T04:32:13.723527+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.381954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.507725+0000","last_change":"2024-04-26T04:32:13.108823+0000","last_active":"2024-04-26T04:32:13.507725+0000","last_peered":"2024-04-26T04:32:13.507725+0000","last_clean":"2024-04-26T04:32:13.507725+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:32:13.507725+0000","last_undegraded":"2024-04-26T04:32:13.507725+0000","last_fullsized":"2024-04-26T04:32:13.507725+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.510373+0000","last_change":"2024-04-26T04:32:12.798464+0000","last_active":"2024-04-26T04:32:13.510373+0000","last_peered":"2024-04-26T04:32:13.510373+0000","last_clean":"2024-04-26T04:32:13.510373+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:32:13.510373+0000","last_undegraded":"2024-04-26T04:32:13.510373+0000","last_fullsized":"2024-04-26T04:32:13.510373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.684262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.512246+0000","last_change":"2024-04-26T04:32:12.505739+0000","last_active":"2024-04-26T04:32:13.512246+0000","last_peered":"2024-04-26T04:32:13.512246+0000","last_clean":"2024-04-26T04:32:13.512246+0000","last_became_active":"2024-04-26T04:32:12.504908+0000","last_became_peered":"2024-04-26T04:32:12.504908+0000","last_unstale":"2024-04-26T04:32:13.512246+0000","last_undegraded":"2024-04-26T04:32:13.512246+0000","last_fullsized":"2024-04-26T04:32:13.512246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:41.080750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.610584+0000","last_change":"2024-04-26T04:32:12.603501+0000","last_active":"2024-04-26T04:32:13.610584+0000","last_peered":"2024-04-26T04:32:13.610584+0000","last_clean":"2024-04-26T04:32:13.610584+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:32:13.610584+0000","last_undegraded":"2024-04-26T04:32:13.610584+0000","last_fullsized":"2024-04-26T04:32:13.610584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.945207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.505930+0000","last_change":"2024-04-26T04:32:12.505010+0000","last_active":"2024-04-26T04:32:13.505930+0000","last_peered":"2024-04-26T04:32:13.505930+0000","last_clean":"2024-04-26T04:32:13.505930+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:32:13.505930+0000","last_undegraded":"2024-04-26T04:32:13.505930+0000","last_fullsized":"2024-04-26T04:32:13.505930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.374592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'60","reported_seq":97,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:15.289057+0000","last_change":"2024-04-26T04:32:10.731528+0000","last_active":"2024-04-26T04:32:15.289057+0000","last_peered":"2024-04-26T04:32:15.289057+0000","last_clean":"2024-04-26T04:32:15.289057+0000","last_became_active":"2024-04-26T04:32:10.729056+0000","last_became_peered":"2024-04-26T04:32:10.729056+0000","last_unstale":"2024-04-26T04:32:15.289057+0000","last_undegraded":"2024-04-26T04:32:15.289057+0000","last_fullsized":"2024-04-26T04:32:15.289057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:32:08.365921+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:10.391321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":62,"num_write":92,"num_write_kb":1109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":62,"num_write":92,"num_write_kb":1109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60,"ondisk_log_size":60,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247813,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":487424,"data_stored":315356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,7,8,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,6,7,8,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,4,7,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,3,4,7,8,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247814,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":487424,"data_stored":315356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,8,9,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,3,4,6,7,8,9,10,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,4,6,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31440,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247814,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:32:17.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:32:17.741 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:17.742 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:32:17.758 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-04-26T04:32:17.325961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":62,"num_write":92,"num_write_kb":1109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60,"ondisk_log_size":60,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":445388,"kb_used_data":4648,"kb_used_omap":24,"kb_used_meta":440423,"kb_avail":1509504052,"statfs":{"total":1546188226560,"available":1545732149248,"internally_reserved":0,"allocated":4759552,"data_stored":2104870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":450993231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131072,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":28,"num_write_kb":522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000702"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.511264+0000","last_change":"2024-04-26T04:32:12.803651+0000","last_active":"2024-04-26T04:32:13.511264+0000","last_peered":"2024-04-26T04:32:13.511264+0000","last_clean":"2024-04-26T04:32:13.511264+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:32:13.511264+0000","last_undegraded":"2024-04-26T04:32:13.511264+0000","last_fullsized":"2024-04-26T04:32:13.511264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:26.614652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.840131+0000","last_change":"2024-04-26T04:32:12.697838+0000","last_active":"2024-04-26T04:32:13.840131+0000","last_peered":"2024-04-26T04:32:13.840131+0000","last_clean":"2024-04-26T04:32:13.840131+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:32:13.840131+0000","last_undegraded":"2024-04-26T04:32:13.840131+0000","last_fullsized":"2024-04-26T04:32:13.840131+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.961816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.723527+0000","last_change":"2024-04-26T04:32:12.504400+0000","last_active":"2024-04-26T04:32:13.723527+0000","last_peered":"2024-04-26T04:32:13.723527+0000","last_clean":"2024-04-26T04:32:13.723527+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:32:13.723527+0000","last_undegraded":"2024-04-26T04:32:13.723527+0000","last_fullsized":"2024-04-26T04:32:13.723527+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.381954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.507725+0000","last_change":"2024-04-26T04:32:13.108823+0000","last_active":"2024-04-26T04:32:13.507725+0000","last_peered":"2024-04-26T04:32:13.507725+0000","last_clean":"2024-04-26T04:32:13.507725+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:32:13.507725+0000","last_undegraded":"2024-04-26T04:32:13.507725+0000","last_fullsized":"2024-04-26T04:32:13.507725+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.510373+0000","last_change":"2024-04-26T04:32:12.798464+0000","last_active":"2024-04-26T04:32:13.510373+0000","last_peered":"2024-04-26T04:32:13.510373+0000","last_clean":"2024-04-26T04:32:13.510373+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:32:13.510373+0000","last_undegraded":"2024-04-26T04:32:13.510373+0000","last_fullsized":"2024-04-26T04:32:13.510373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.684262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.512246+0000","last_change":"2024-04-26T04:32:12.505739+0000","last_active":"2024-04-26T04:32:13.512246+0000","last_peered":"2024-04-26T04:32:13.512246+0000","last_clean":"2024-04-26T04:32:13.512246+0000","last_became_active":"2024-04-26T04:32:12.504908+0000","last_became_peered":"2024-04-26T04:32:12.504908+0000","last_unstale":"2024-04-26T04:32:13.512246+0000","last_undegraded":"2024-04-26T04:32:13.512246+0000","last_fullsized":"2024-04-26T04:32:13.512246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:41.080750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.610584+0000","last_change":"2024-04-26T04:32:12.603501+0000","last_active":"2024-04-26T04:32:13.610584+0000","last_peered":"2024-04-26T04:32:13.610584+0000","last_clean":"2024-04-26T04:32:13.610584+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:32:13.610584+0000","last_undegraded":"2024-04-26T04:32:13.610584+0000","last_fullsized":"2024-04-26T04:32:13.610584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.945207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.505930+0000","last_change":"2024-04-26T04:32:12.505010+0000","last_active":"2024-04-26T04:32:13.505930+0000","last_peered":"2024-04-26T04:32:13.505930+0000","last_clean":"2024-04-26T04:32:13.505930+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:32:13.505930+0000","last_undegraded":"2024-04-26T04:32:13.505930+0000","last_fullsized":"2024-04-26T04:32:13.505930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.374592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'60","reported_seq":97,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:15.289057+0000","last_change":"2024-04-26T04:32:10.731528+0000","last_active":"2024-04-26T04:32:15.289057+0000","last_peered":"2024-04-26T04:32:15.289057+0000","last_clean":"2024-04-26T04:32:15.289057+0000","last_became_active":"2024-04-26T04:32:10.729056+0000","last_became_peered":"2024-04-26T04:32:10.729056+0000","last_unstale":"2024-04-26T04:32:15.289057+0000","last_undegraded":"2024-04-26T04:32:15.289057+0000","last_fullsized":"2024-04-26T04:32:15.289057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:32:08.365921+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:10.391321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":62,"num_write":92,"num_write_kb":1109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":62,"num_write":92,"num_write_kb":1109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60,"ondisk_log_size":60,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247813,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":487424,"data_stored":315356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,7,8,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,6,7,8,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,4,7,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,3,4,7,8,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247814,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":487424,"data_stored":315356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,8,9,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,3,4,6,7,8,9,10,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,4,6,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31440,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247814,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:32:17.760 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T04:32:17.760 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T04:32:17.760 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T04:32:17.760 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T04:32:18.083 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:18.083 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T04:32:18.101 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T04:32:18.101 INFO:teuthology.run_tasks:Running task thrashosds... 2024-04-26T04:32:18.111 INFO:tasks.thrashosds:config is {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-04-26T04:32:18.112 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuri2-testing-2024-04-24-0914-squid'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1, 'osd_object_clean_region_max_num_intervals': 1000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.1, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri2-testing-2024-04-24-0914-squid', 'sha1': 'ddeaa91797339a19172c3036ff48cca58c12f448'}} 2024-04-26T04:32:18.112 INFO:tasks.thrashosds:config is {'bluestore_new_sharding': 'random', 'chance_bluestore_reshard': 1, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-04-26T04:32:18.112 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-04-26T04:32:18.418 INFO:tasks.thrashosds:Beginning thrashosds... 2024-04-26T04:32:18.418 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T04:32:18.418 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:32:18.711 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:18.711 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:32:18.728 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-04-26T04:32:17.325961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":62,"num_write":92,"num_write_kb":1109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60,"ondisk_log_size":60,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":445388,"kb_used_data":4648,"kb_used_omap":24,"kb_used_meta":440423,"kb_avail":1509504052,"statfs":{"total":1546188226560,"available":1545732149248,"internally_reserved":0,"allocated":4759552,"data_stored":2104870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":450993231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131072,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":28,"num_write_kb":522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000702"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.511264+0000","last_change":"2024-04-26T04:32:12.803651+0000","last_active":"2024-04-26T04:32:13.511264+0000","last_peered":"2024-04-26T04:32:13.511264+0000","last_clean":"2024-04-26T04:32:13.511264+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:32:13.511264+0000","last_undegraded":"2024-04-26T04:32:13.511264+0000","last_fullsized":"2024-04-26T04:32:13.511264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:26.614652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.840131+0000","last_change":"2024-04-26T04:32:12.697838+0000","last_active":"2024-04-26T04:32:13.840131+0000","last_peered":"2024-04-26T04:32:13.840131+0000","last_clean":"2024-04-26T04:32:13.840131+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:32:13.840131+0000","last_undegraded":"2024-04-26T04:32:13.840131+0000","last_fullsized":"2024-04-26T04:32:13.840131+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.961816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.723527+0000","last_change":"2024-04-26T04:32:12.504400+0000","last_active":"2024-04-26T04:32:13.723527+0000","last_peered":"2024-04-26T04:32:13.723527+0000","last_clean":"2024-04-26T04:32:13.723527+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:32:13.723527+0000","last_undegraded":"2024-04-26T04:32:13.723527+0000","last_fullsized":"2024-04-26T04:32:13.723527+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.381954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.507725+0000","last_change":"2024-04-26T04:32:13.108823+0000","last_active":"2024-04-26T04:32:13.507725+0000","last_peered":"2024-04-26T04:32:13.507725+0000","last_clean":"2024-04-26T04:32:13.507725+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:32:13.507725+0000","last_undegraded":"2024-04-26T04:32:13.507725+0000","last_fullsized":"2024-04-26T04:32:13.507725+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.510373+0000","last_change":"2024-04-26T04:32:12.798464+0000","last_active":"2024-04-26T04:32:13.510373+0000","last_peered":"2024-04-26T04:32:13.510373+0000","last_clean":"2024-04-26T04:32:13.510373+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:32:13.510373+0000","last_undegraded":"2024-04-26T04:32:13.510373+0000","last_fullsized":"2024-04-26T04:32:13.510373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.684262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.512246+0000","last_change":"2024-04-26T04:32:12.505739+0000","last_active":"2024-04-26T04:32:13.512246+0000","last_peered":"2024-04-26T04:32:13.512246+0000","last_clean":"2024-04-26T04:32:13.512246+0000","last_became_active":"2024-04-26T04:32:12.504908+0000","last_became_peered":"2024-04-26T04:32:12.504908+0000","last_unstale":"2024-04-26T04:32:13.512246+0000","last_undegraded":"2024-04-26T04:32:13.512246+0000","last_fullsized":"2024-04-26T04:32:13.512246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:41.080750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.610584+0000","last_change":"2024-04-26T04:32:12.603501+0000","last_active":"2024-04-26T04:32:13.610584+0000","last_peered":"2024-04-26T04:32:13.610584+0000","last_clean":"2024-04-26T04:32:13.610584+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:32:13.610584+0000","last_undegraded":"2024-04-26T04:32:13.610584+0000","last_fullsized":"2024-04-26T04:32:13.610584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.945207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.505930+0000","last_change":"2024-04-26T04:32:12.505010+0000","last_active":"2024-04-26T04:32:13.505930+0000","last_peered":"2024-04-26T04:32:13.505930+0000","last_clean":"2024-04-26T04:32:13.505930+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:32:13.505930+0000","last_undegraded":"2024-04-26T04:32:13.505930+0000","last_fullsized":"2024-04-26T04:32:13.505930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.374592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'60","reported_seq":97,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:15.289057+0000","last_change":"2024-04-26T04:32:10.731528+0000","last_active":"2024-04-26T04:32:15.289057+0000","last_peered":"2024-04-26T04:32:15.289057+0000","last_clean":"2024-04-26T04:32:15.289057+0000","last_became_active":"2024-04-26T04:32:10.729056+0000","last_became_peered":"2024-04-26T04:32:10.729056+0000","last_unstale":"2024-04-26T04:32:15.289057+0000","last_undegraded":"2024-04-26T04:32:15.289057+0000","last_fullsized":"2024-04-26T04:32:15.289057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:32:08.365921+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:10.391321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":62,"num_write":92,"num_write_kb":1109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":62,"num_write":92,"num_write_kb":1109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60,"ondisk_log_size":60,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247813,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":487424,"data_stored":315356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,7,8,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,6,7,8,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,4,7,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,3,4,7,8,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247814,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":487424,"data_stored":315356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,8,9,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,3,4,6,7,8,9,10,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,4,6,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31440,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247814,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:32:18.729 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:32:19.029 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:19.029 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:32:19.047 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-04-26T04:32:17.325961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":62,"num_write":92,"num_write_kb":1109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60,"ondisk_log_size":60,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":445388,"kb_used_data":4648,"kb_used_omap":24,"kb_used_meta":440423,"kb_avail":1509504052,"statfs":{"total":1546188226560,"available":1545732149248,"internally_reserved":0,"allocated":4759552,"data_stored":2104870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":450993231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131072,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":28,"num_write_kb":522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000702"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.511264+0000","last_change":"2024-04-26T04:32:12.803651+0000","last_active":"2024-04-26T04:32:13.511264+0000","last_peered":"2024-04-26T04:32:13.511264+0000","last_clean":"2024-04-26T04:32:13.511264+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:32:13.511264+0000","last_undegraded":"2024-04-26T04:32:13.511264+0000","last_fullsized":"2024-04-26T04:32:13.511264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:26.614652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.840131+0000","last_change":"2024-04-26T04:32:12.697838+0000","last_active":"2024-04-26T04:32:13.840131+0000","last_peered":"2024-04-26T04:32:13.840131+0000","last_clean":"2024-04-26T04:32:13.840131+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:32:13.840131+0000","last_undegraded":"2024-04-26T04:32:13.840131+0000","last_fullsized":"2024-04-26T04:32:13.840131+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.961816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.723527+0000","last_change":"2024-04-26T04:32:12.504400+0000","last_active":"2024-04-26T04:32:13.723527+0000","last_peered":"2024-04-26T04:32:13.723527+0000","last_clean":"2024-04-26T04:32:13.723527+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:32:13.723527+0000","last_undegraded":"2024-04-26T04:32:13.723527+0000","last_fullsized":"2024-04-26T04:32:13.723527+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.381954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.507725+0000","last_change":"2024-04-26T04:32:13.108823+0000","last_active":"2024-04-26T04:32:13.507725+0000","last_peered":"2024-04-26T04:32:13.507725+0000","last_clean":"2024-04-26T04:32:13.507725+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:32:13.507725+0000","last_undegraded":"2024-04-26T04:32:13.507725+0000","last_fullsized":"2024-04-26T04:32:13.507725+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.510373+0000","last_change":"2024-04-26T04:32:12.798464+0000","last_active":"2024-04-26T04:32:13.510373+0000","last_peered":"2024-04-26T04:32:13.510373+0000","last_clean":"2024-04-26T04:32:13.510373+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:32:13.510373+0000","last_undegraded":"2024-04-26T04:32:13.510373+0000","last_fullsized":"2024-04-26T04:32:13.510373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.684262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.512246+0000","last_change":"2024-04-26T04:32:12.505739+0000","last_active":"2024-04-26T04:32:13.512246+0000","last_peered":"2024-04-26T04:32:13.512246+0000","last_clean":"2024-04-26T04:32:13.512246+0000","last_became_active":"2024-04-26T04:32:12.504908+0000","last_became_peered":"2024-04-26T04:32:12.504908+0000","last_unstale":"2024-04-26T04:32:13.512246+0000","last_undegraded":"2024-04-26T04:32:13.512246+0000","last_fullsized":"2024-04-26T04:32:13.512246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:41.080750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.610584+0000","last_change":"2024-04-26T04:32:12.603501+0000","last_active":"2024-04-26T04:32:13.610584+0000","last_peered":"2024-04-26T04:32:13.610584+0000","last_clean":"2024-04-26T04:32:13.610584+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:32:13.610584+0000","last_undegraded":"2024-04-26T04:32:13.610584+0000","last_fullsized":"2024-04-26T04:32:13.610584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.945207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.505930+0000","last_change":"2024-04-26T04:32:12.505010+0000","last_active":"2024-04-26T04:32:13.505930+0000","last_peered":"2024-04-26T04:32:13.505930+0000","last_clean":"2024-04-26T04:32:13.505930+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:32:13.505930+0000","last_undegraded":"2024-04-26T04:32:13.505930+0000","last_fullsized":"2024-04-26T04:32:13.505930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.374592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'60","reported_seq":97,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:15.289057+0000","last_change":"2024-04-26T04:32:10.731528+0000","last_active":"2024-04-26T04:32:15.289057+0000","last_peered":"2024-04-26T04:32:15.289057+0000","last_clean":"2024-04-26T04:32:15.289057+0000","last_became_active":"2024-04-26T04:32:10.729056+0000","last_became_peered":"2024-04-26T04:32:10.729056+0000","last_unstale":"2024-04-26T04:32:15.289057+0000","last_undegraded":"2024-04-26T04:32:15.289057+0000","last_fullsized":"2024-04-26T04:32:15.289057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:32:08.365921+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:10.391321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":62,"num_write":92,"num_write_kb":1109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":62,"num_write":92,"num_write_kb":1109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60,"ondisk_log_size":60,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247813,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":487424,"data_stored":315356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,7,8,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,6,7,8,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,4,7,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,3,4,7,8,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313220,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,3,4,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247814,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":487424,"data_stored":315356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,7,8,9,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31464,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,3,4,6,7,8,9,10,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247813,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,4,6,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31440,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":262144,"data_stored":100566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247814,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:32:19.048 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T04:32:19.048 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-04-26T04:32:19.336 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 27 2024-04-26T04:32:19.336 INFO:teuthology.orchestra.run.smithi012.stdout:fsid 12c27fbe-0161-48d0-88ec-42a226685437 2024-04-26T04:32:19.336 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-04-26T04:31:56.451062+0000 2024-04-26T04:32:19.336 INFO:teuthology.orchestra.run.smithi012.stdout:modified 2024-04-26T04:32:13.489645+0000 2024-04-26T04:32:19.337 INFO:teuthology.orchestra.run.smithi012.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T04:32:19.337 INFO:teuthology.orchestra.run.smithi012.stdout:crush_version 5 2024-04-26T04:32:19.337 INFO:teuthology.orchestra.run.smithi012.stdout:full_ratio 0.95 2024-04-26T04:32:19.337 INFO:teuthology.orchestra.run.smithi012.stdout:backfillfull_ratio 0.9 2024-04-26T04:32:19.337 INFO:teuthology.orchestra.run.smithi012.stdout:nearfull_ratio 0.85 2024-04-26T04:32:19.337 INFO:teuthology.orchestra.run.smithi012.stdout:require_min_compat_client luminous 2024-04-26T04:32:19.337 INFO:teuthology.orchestra.run.smithi012.stdout:min_compat_client jewel 2024-04-26T04:32:19.337 INFO:teuthology.orchestra.run.smithi012.stdout:require_osd_release squid 2024-04-26T04:32:19.337 INFO:teuthology.orchestra.run.smithi012.stdout:stretch_mode_enabled false 2024-04-26T04:32:19.337 INFO:teuthology.orchestra.run.smithi012.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 25 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 15.38 2024-04-26T04:32:19.337 INFO:teuthology.orchestra.run.smithi012.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 27 flags hashpspool stripe_width 0 application rbd read_balance_score 4.02 2024-04-26T04:32:19.337 INFO:teuthology.orchestra.run.smithi012.stdout:max_osd 16 2024-04-26T04:32:19.337 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6816/1754063127,v1:172.21.15.12:6817/1754063127] [v2:172.21.15.12:6818/1754063127,v1:172.21.15.12:6819/1754063127] exists,up 55a4f254-2c2a-4007-9c52-b89df6fffb69 2024-04-26T04:32:19.338 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6808/3264326676,v1:172.21.15.113:6809/3264326676] [v2:172.21.15.113:6810/3264326676,v1:172.21.15.113:6811/3264326676] exists,up 32db4f97-c8ac-481c-9e80-18fc91004298 2024-04-26T04:32:19.338 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6800/1454475448,v1:172.21.15.137:6802/1454475448] [v2:172.21.15.137:6804/1454475448,v1:172.21.15.137:6806/1454475448] exists,up 2cd26466-5d59-4599-ba3f-a78b0e3b681f 2024-04-26T04:32:19.338 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6803/2898698985,v1:172.21.15.178:6805/2898698985] [v2:172.21.15.178:6808/2898698985,v1:172.21.15.178:6810/2898698985] exists,up 0945017d-9525-49d6-aa67-6635acd94c3e 2024-04-26T04:32:19.338 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6808/3209840823,v1:172.21.15.12:6809/3209840823] [v2:172.21.15.12:6810/3209840823,v1:172.21.15.12:6811/3209840823] exists,up 38dc68bf-0a89-4900-9d35-24c70368447a 2024-04-26T04:32:19.338 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6800/1475637773,v1:172.21.15.113:6801/1475637773] [v2:172.21.15.113:6802/1475637773,v1:172.21.15.113:6803/1475637773] exists,up e91cc8a1-57e1-4760-a329-8a60aa752a95 2024-04-26T04:32:19.338 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6816/771338783,v1:172.21.15.137:6817/771338783] [v2:172.21.15.137:6818/771338783,v1:172.21.15.137:6819/771338783] exists,up a1322130-3391-432d-a1ee-f83297885029 2024-04-26T04:32:19.338 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6826/1017152077,v1:172.21.15.178:6827/1017152077] [v2:172.21.15.178:6828/1017152077,v1:172.21.15.178:6829/1017152077] exists,up 02ee4ca2-9d48-4aff-9641-1823760c7a3b 2024-04-26T04:32:19.338 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6800/2153129138,v1:172.21.15.12:6801/2153129138] [v2:172.21.15.12:6802/2153129138,v1:172.21.15.12:6803/2153129138] exists,up ea3d5f62-3d68-4824-8556-da8db36553fb 2024-04-26T04:32:19.338 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6824/359931551,v1:172.21.15.113:6825/359931551] [v2:172.21.15.113:6826/359931551,v1:172.21.15.113:6827/359931551] exists,up e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70 2024-04-26T04:32:19.338 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6801/1533480861,v1:172.21.15.137:6803/1533480861] [v2:172.21.15.137:6805/1533480861,v1:172.21.15.137:6807/1533480861] exists,up 8c4f8df5-465a-486e-9413-1de14e33c119 2024-04-26T04:32:19.338 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6818/1683682957,v1:172.21.15.178:6819/1683682957] [v2:172.21.15.178:6820/1683682957,v1:172.21.15.178:6821/1683682957] exists,up 6fe54c2d-0b95-4235-8c22-51230cfaa591 2024-04-26T04:32:19.338 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6824/2509311840,v1:172.21.15.12:6825/2509311840] [v2:172.21.15.12:6826/2509311840,v1:172.21.15.12:6827/2509311840] exists,up 5ddd0f05-fb78-43bb-95ec-3fc7985603b3 2024-04-26T04:32:19.338 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6815/3716239663,v1:172.21.15.113:6816/3716239663] [v2:172.21.15.113:6818/3716239663,v1:172.21.15.113:6819/3716239663] exists,up 37eb42c1-f29e-49f5-b8f6-035178038e5d 2024-04-26T04:32:19.339 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6824/1454956132,v1:172.21.15.137:6825/1454956132] [v2:172.21.15.137:6826/1454956132,v1:172.21.15.137:6827/1454956132] exists,up b2267367-a857-4a5c-a8d1-6a9b23c748f6 2024-04-26T04:32:19.339 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6802/314165740,v1:172.21.15.178:6804/314165740] [v2:172.21.15.178:6806/314165740,v1:172.21.15.178:6807/314165740] exists,up 9933b5aa-e091-43b1-b788-5b79695bab11 2024-04-26T04:32:19.354 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6816/1754063127,v1:172.21.15.12:6817/1754063127] [v2:172.21.15.12:6818/1754063127,v1:172.21.15.12:6819/1754063127] exists,up 55a4f254-2c2a-4007-9c52-b89df6fffb69', 'osd.1 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6808/3264326676,v1:172.21.15.113:6809/3264326676] [v2:172.21.15.113:6810/3264326676,v1:172.21.15.113:6811/3264326676] exists,up 32db4f97-c8ac-481c-9e80-18fc91004298', 'osd.2 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6800/1454475448,v1:172.21.15.137:6802/1454475448] [v2:172.21.15.137:6804/1454475448,v1:172.21.15.137:6806/1454475448] exists,up 2cd26466-5d59-4599-ba3f-a78b0e3b681f', 'osd.3 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6803/2898698985,v1:172.21.15.178:6805/2898698985] [v2:172.21.15.178:6808/2898698985,v1:172.21.15.178:6810/2898698985] exists,up 0945017d-9525-49d6-aa67-6635acd94c3e', 'osd.4 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6808/3209840823,v1:172.21.15.12:6809/3209840823] [v2:172.21.15.12:6810/3209840823,v1:172.21.15.12:6811/3209840823] exists,up 38dc68bf-0a89-4900-9d35-24c70368447a', 'osd.5 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6800/1475637773,v1:172.21.15.113:6801/1475637773] [v2:172.21.15.113:6802/1475637773,v1:172.21.15.113:6803/1475637773] exists,up e91cc8a1-57e1-4760-a329-8a60aa752a95', 'osd.6 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6816/771338783,v1:172.21.15.137:6817/771338783] [v2:172.21.15.137:6818/771338783,v1:172.21.15.137:6819/771338783] exists,up a1322130-3391-432d-a1ee-f83297885029', 'osd.7 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6826/1017152077,v1:172.21.15.178:6827/1017152077] [v2:172.21.15.178:6828/1017152077,v1:172.21.15.178:6829/1017152077] exists,up 02ee4ca2-9d48-4aff-9641-1823760c7a3b', 'osd.8 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6800/2153129138,v1:172.21.15.12:6801/2153129138] [v2:172.21.15.12:6802/2153129138,v1:172.21.15.12:6803/2153129138] exists,up ea3d5f62-3d68-4824-8556-da8db36553fb', 'osd.9 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6824/359931551,v1:172.21.15.113:6825/359931551] [v2:172.21.15.113:6826/359931551,v1:172.21.15.113:6827/359931551] exists,up e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70', 'osd.10 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6801/1533480861,v1:172.21.15.137:6803/1533480861] [v2:172.21.15.137:6805/1533480861,v1:172.21.15.137:6807/1533480861] exists,up 8c4f8df5-465a-486e-9413-1de14e33c119', 'osd.11 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6818/1683682957,v1:172.21.15.178:6819/1683682957] [v2:172.21.15.178:6820/1683682957,v1:172.21.15.178:6821/1683682957] exists,up 6fe54c2d-0b95-4235-8c22-51230cfaa591', 'osd.12 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.12:6824/2509311840,v1:172.21.15.12:6825/2509311840] [v2:172.21.15.12:6826/2509311840,v1:172.21.15.12:6827/2509311840] exists,up 5ddd0f05-fb78-43bb-95ec-3fc7985603b3', 'osd.13 up in weight 1 up_from 23 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6815/3716239663,v1:172.21.15.113:6816/3716239663] [v2:172.21.15.113:6818/3716239663,v1:172.21.15.113:6819/3716239663] exists,up 37eb42c1-f29e-49f5-b8f6-035178038e5d', 'osd.14 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6824/1454956132,v1:172.21.15.137:6825/1454956132] [v2:172.21.15.137:6826/1454956132,v1:172.21.15.137:6827/1454956132] exists,up b2267367-a857-4a5c-a8d1-6a9b23c748f6', 'osd.15 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6802/314165740,v1:172.21.15.178:6804/314165740] [v2:172.21.15.178:6806/314165740,v1:172.21.15.178:6807/314165740] exists,up 9933b5aa-e091-43b1-b788-5b79695bab11'] 2024-04-26T04:32:19.355 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-04-26T04:32:19.470 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:19.471 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "mon.a", 2024-04-26T04:32:19.471 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster": "ceph", 2024-04-26T04:32:19.471 INFO:teuthology.orchestra.run.smithi012.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-04-26T04:32:19.471 INFO:teuthology.orchestra.run.smithi012.stdout: "admin_socket_mode": "", 2024-04-26T04:32:19.471 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-04-26T04:32:19.471 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_client_required": "cephx, none", 2024-04-26T04:32:19.471 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_cluster_required": "cephx", 2024-04-26T04:32:19.471 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_debug": "true", 2024-04-26T04:32:19.471 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-04-26T04:32:19.471 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-04-26T04:32:19.471 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_service_required": "cephx", 2024-04-26T04:32:19.471 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_service_ticket_ttl": "240.000000", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "auth_supported": "cephx", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_aio": "true", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_aio_max_queue_depth": "1024", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_aio_poll_ms": "250", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_aio_reap_max": "16", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_async_discard": "false", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_block_size": "4096", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_debug_aio": "false", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_debug_inflight_ios": "false", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_enable_discard": "false", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_flock_retry": "3", 2024-04-26T04:32:19.472 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_flock_retry_interval": "0.100000", 2024-04-26T04:32:19.473 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_inject_crash": "0", 2024-04-26T04:32:19.473 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_inject_crash_flush_delay": "2", 2024-04-26T04:32:19.473 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_ioring": "false", 2024-04-26T04:32:19.473 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_ioring_hipri": "false", 2024-04-26T04:32:19.473 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_ioring_sqthread_poll": "false", 2024-04-26T04:32:19.473 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-04-26T04:32:19.473 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_read_buffer_alignment": "4096", 2024-04-26T04:32:19.474 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-04-26T04:32:19.474 INFO:teuthology.orchestra.run.smithi012.stdout: "bdev_type": "", 2024-04-26T04:32:19.474 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_alloc_size": "1048576", 2024-04-26T04:32:19.474 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_allocator": "hybrid", 2024-04-26T04:32:19.474 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_buffered_io": "true", 2024-04-26T04:32:19.474 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_check_for_zeros": "false", 2024-04-26T04:32:19.474 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_check_volume_selector_often": "false", 2024-04-26T04:32:19.474 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_compact_log_sync": "false", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_log_compact_min_size": "16777216", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_log_replay_check_allocations": "true", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_max_log_runway": "4194304", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_max_prefetch": "1048576", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_min_flush_size": "524288", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_min_log_runway": "1048576", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_replay_recovery": "false", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_shared_alloc_size": "65536", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluefs_sync_write": "false", 2024-04-26T04:32:19.475 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-04-26T04:32:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-04-26T04:32:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-04-26T04:32:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_allocation_from_file": "true", 2024-04-26T04:32:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_allocator": "hybrid", 2024-04-26T04:32:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-04-26T04:32:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-04-26T04:32:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-04-26T04:32:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-04-26T04:32:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-04-26T04:32:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-04-26T04:32:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_blobid_prealloc": "10240", 2024-04-26T04:32:19.476 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_create": "true", 2024-04-26T04:32:19.477 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_db_create": "false", 2024-04-26T04:32:19.477 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_db_path": "", 2024-04-26T04:32:19.477 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_db_size": "0", 2024-04-26T04:32:19.477 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_path": "", 2024-04-26T04:32:19.477 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_preallocate_file": "false", 2024-04-26T04:32:19.477 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_size": "107374182400", 2024-04-26T04:32:19.477 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_wal_create": "false", 2024-04-26T04:32:19.477 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_wal_path": "", 2024-04-26T04:32:19.477 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_block_wal_size": "100663296", 2024-04-26T04:32:19.477 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_bluefs": "true", 2024-04-26T04:32:19.477 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-04-26T04:32:19.477 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_bluefs_env_mirror": "false", 2024-04-26T04:32:19.477 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-04-26T04:32:19.478 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-04-26T04:32:19.478 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-04-26T04:32:19.478 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-04-26T04:32:19.478 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-04-26T04:32:19.478 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-04-26T04:32:19.478 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_autotune": "true", 2024-04-26T04:32:19.478 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-04-26T04:32:19.478 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-04-26T04:32:19.478 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-04-26T04:32:19.478 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-04-26T04:32:19.478 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_size": "0", 2024-04-26T04:32:19.479 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-04-26T04:32:19.479 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-04-26T04:32:19.479 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-04-26T04:32:19.479 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-04-26T04:32:19.479 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cache_type": "2q", 2024-04-26T04:32:19.479 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-04-26T04:32:19.479 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_clone_cow": "true", 2024-04-26T04:32:19.479 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_algorithm": "snappy", 2024-04-26T04:32:19.479 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_max_blob_size": "0", 2024-04-26T04:32:19.479 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-04-26T04:32:19.479 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-04-26T04:32:19.479 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_min_blob_size": "0", 2024-04-26T04:32:19.479 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_mode": "none", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_csum_type": "crc32c", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_enforce_settings": "default", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_freelist": "false", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_fsck_abort": "false", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_inject_read_err": "false", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_legacy_omap": "false", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-04-26T04:32:19.480 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_prefill": "0.000000", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_small_allocations": "0", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_default_buffered_read": "true", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_default_buffered_write": "false", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_deferred_batch_ops": "0", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_elastic_shared_blobs": "true", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-04-26T04:32:19.481 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-04-26T04:32:19.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-04-26T04:32:19.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-04-26T04:32:19.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-04-26T04:32:19.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fail_eio": "false", 2024-04-26T04:32:19.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-04-26T04:32:19.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-04-26T04:32:19.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-04-26T04:32:19.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-04-26T04:32:19.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_on_mkfs": "true", 2024-04-26T04:32:19.482 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-04-26T04:32:19.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_on_mount": "false", 2024-04-26T04:32:19.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-04-26T04:32:19.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_on_umount": "false", 2024-04-26T04:32:19.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-04-26T04:32:19.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-04-26T04:32:19.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-04-26T04:32:19.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-04-26T04:32:19.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-04-26T04:32:19.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-04-26T04:32:19.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-04-26T04:32:19.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-04-26T04:32:19.483 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_ignore_data_csum": "false", 2024-04-26T04:32:19.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-04-26T04:32:19.484 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_kvbackend": "rocksdb", 2024-04-26T04:32:19.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-04-26T04:32:19.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-04-26T04:32:19.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_log_op_age": "5.000000", 2024-04-26T04:32:19.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_max_alloc_size": "0", 2024-04-26T04:32:19.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_max_blob_size": "0", 2024-04-26T04:32:19.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-04-26T04:32:19.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-04-26T04:32:19.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_max_defer_interval": "3.000000", 2024-04-26T04:32:19.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_max_deferred_txc": "32", 2024-04-26T04:32:19.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_min_alloc_size": "0", 2024-04-26T04:32:19.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-04-26T04:32:19.485 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-04-26T04:32:19.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_nid_prealloc": "1024", 2024-04-26T04:32:19.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_prefer_deferred_size": "0", 2024-04-26T04:32:19.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-04-26T04:32:19.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-04-26T04:32:19.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_qfsck_on_mount": "true", 2024-04-26T04:32:19.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_retry_disk_reads": "3", 2024-04-26T04:32:19.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_rocksdb_cf": "true", 2024-04-26T04:32:19.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-04-26T04:32:19.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-04-26T04:32:19.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_rocksdb_options_annex": "", 2024-04-26T04:32:19.486 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_spdk_coremask": "0x1", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_spdk_io_sleep": "5", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_spdk_max_io_completion": "0", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_spdk_mem": "512", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_sync_submit_transaction": "false", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_throttle_bytes": "67108864", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_throttle_cost_per_io": "0", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_tracing": "false", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-04-26T04:32:19.487 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_volume_selection_reserved": "0", 2024-04-26T04:32:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-04-26T04:32:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-04-26T04:32:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-04-26T04:32:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-04-26T04:32:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-04-26T04:32:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-04-26T04:32:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: "bluestore_zero_block_detection": "false", 2024-04-26T04:32:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-04-26T04:32:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_action_update_interval": "2", 2024-04-26T04:32:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-04-26T04:32:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-04-26T04:32:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-04-26T04:32:19.488 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_mount_timeout": "10", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_cluster_require_signatures": "false", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_cluster_require_version": "2", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_require_signatures": "false", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_require_version": "2", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_service_require_signatures": "false", 2024-04-26T04:32:19.489 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_service_require_version": "2", 2024-04-26T04:32:19.490 INFO:teuthology.orchestra.run.smithi012.stdout: "cephx_sign_messages": "true", 2024-04-26T04:32:19.490 INFO:teuthology.orchestra.run.smithi012.stdout: "chdir": "", 2024-04-26T04:32:19.490 INFO:teuthology.orchestra.run.smithi012.stdout: "client_acl_type": "", 2024-04-26T04:32:19.490 INFO:teuthology.orchestra.run.smithi012.stdout: "client_asio_thread_count": "2", 2024-04-26T04:32:19.490 INFO:teuthology.orchestra.run.smithi012.stdout: "client_cache_mid": "0.750000", 2024-04-26T04:32:19.490 INFO:teuthology.orchestra.run.smithi012.stdout: "client_cache_size": "16384", 2024-04-26T04:32:19.490 INFO:teuthology.orchestra.run.smithi012.stdout: "client_caps_release_delay": "5", 2024-04-26T04:32:19.490 INFO:teuthology.orchestra.run.smithi012.stdout: "client_check_pool_perm": "true", 2024-04-26T04:32:19.542 INFO:teuthology.orchestra.run.smithi012.stdout: "client_collect_and_send_global_metrics": "false", 2024-04-26T04:32:19.542 INFO:teuthology.orchestra.run.smithi012.stdout: "client_debug_force_sync_read": "false", 2024-04-26T04:32:19.542 INFO:teuthology.orchestra.run.smithi012.stdout: "client_debug_getattr_caps": "false", 2024-04-26T04:32:19.542 INFO:teuthology.orchestra.run.smithi012.stdout: "client_debug_inject_tick_delay": "0", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_die_on_failed_remount": "false", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_dirsize_rbytes": "true", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_force_lazyio": "false", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_fs": "", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_inject_fixed_oldest_tid": "false", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_inject_release_failure": "false", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_max_inline_size": "4096", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_max_retries_on_remount_failure": "5", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_mds_namespace": "", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_metadata": "", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_mount_gid": "-1", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_mount_timeout": "300", 2024-04-26T04:32:19.543 INFO:teuthology.orchestra.run.smithi012.stdout: "client_mount_uid": "-1", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_mountpoint": "/", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_notify_timeout": "10", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_oc": "true", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_oc_max_dirty": "104857600", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_oc_max_dirty_age": "5.000000", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_oc_max_objects": "1000", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_oc_size": "209715200", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_oc_target_dirty": "8388608", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_permissions": "true", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_quota": "true", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_quota_df": "true", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_readahead_max_bytes": "0", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_readahead_max_periods": "4", 2024-04-26T04:32:19.544 INFO:teuthology.orchestra.run.smithi012.stdout: "client_readahead_min": "131072", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client_reconnect_stale": "false", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client_shutdown_timeout": "30", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client_snapdir": ".snap", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client_tick_interval": "1", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client_trace": "", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client_try_dentry_invalidate": "false", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client_use_faked_inos": "false", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "client_use_random_mds": "false", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_graylog": "false", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_graylog_port": "12201", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_monitors": "default=true", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_syslog": "false", 2024-04-26T04:32:19.545 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-04-26T04:32:19.546 INFO:teuthology.orchestra.run.smithi012.stdout: "clog_to_syslog_level": "info", 2024-04-26T04:32:19.546 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "-", 2024-04-26T04:32:19.546 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_network": "", 2024-04-26T04:32:19.546 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_network_interface": "", 2024-04-26T04:32:19.546 INFO:teuthology.orchestra.run.smithi012.stdout: "compressor_zlib_isal": "false", 2024-04-26T04:32:19.546 INFO:teuthology.orchestra.run.smithi012.stdout: "compressor_zlib_level": "5", 2024-04-26T04:32:19.546 INFO:teuthology.orchestra.run.smithi012.stdout: "compressor_zlib_winsize": "-15", 2024-04-26T04:32:19.546 INFO:teuthology.orchestra.run.smithi012.stdout: "compressor_zstd_level": "1", 2024-04-26T04:32:19.546 INFO:teuthology.orchestra.run.smithi012.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-04-26T04:32:19.546 INFO:teuthology.orchestra.run.smithi012.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-04-26T04:32:19.546 INFO:teuthology.orchestra.run.smithi012.stdout: "crimson_alien_op_num_threads": "6", 2024-04-26T04:32:19.546 INFO:teuthology.orchestra.run.smithi012.stdout: "crimson_alien_thread_cpu_cores": "", 2024-04-26T04:32:19.546 INFO:teuthology.orchestra.run.smithi012.stdout: "crimson_osd_obc_lru_size": "512", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "crimson_seastar_cpu_cores": "", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_location": "", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_location_hook": "", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_location_hook_timeout": "10", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "daemonize": "false", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "daos_pool": "tank", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "dbstore_db_name_prefix": "dbstore", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_alienstore": "0/5", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_allow_any_pool_priority": "false", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_asok": "1/5", 2024-04-26T04:32:19.547 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_asok_assert_abort": "false", 2024-04-26T04:32:19.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_asserts_on_shutdown": "true", 2024-04-26T04:32:19.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_auth": "20/20", 2024-04-26T04:32:19.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_bdev": "1/3", 2024-04-26T04:32:19.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_bluefs": "1/5", 2024-04-26T04:32:19.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_bluestore": "1/5", 2024-04-26T04:32:19.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_buffer": "0/1", 2024-04-26T04:32:19.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_ceph_exporter": "1/5", 2024-04-26T04:32:19.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_cephfs_mirror": "0/5", 2024-04-26T04:32:19.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_cephsqlite": "0/5", 2024-04-26T04:32:19.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_client": "0/5", 2024-04-26T04:32:19.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_compressor": "1/5", 2024-04-26T04:32:19.548 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_context": "0/1", 2024-04-26T04:32:19.549 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_crimson_interrupt": "0/5", 2024-04-26T04:32:19.549 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_crush": "1/1", 2024-04-26T04:32:19.549 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_crypto": "1/5", 2024-04-26T04:32:19.549 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_cyanstore": "0/5", 2024-04-26T04:32:19.549 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_deliberately_leak_memory": "false", 2024-04-26T04:32:19.549 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_disable_randomized_ping": "false", 2024-04-26T04:32:19.549 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_dpdk": "1/5", 2024-04-26T04:32:19.549 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_eventtrace": "1/5", 2024-04-26T04:32:19.549 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_filer": "0/1", 2024-04-26T04:32:19.549 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_filestore": "1/3", 2024-04-26T04:32:19.549 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_finisher": "1/1", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_fuse": "1/5", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_heartbeat_testing_span": "0", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_heartbeatmap": "1/5", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_immutable_obj_cache": "0/5", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_javaclient": "1/5", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_journal": "1/3", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_journaler": "0/5", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_kstore": "1/5", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_lockdep": "0/1", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mclock": "1/5", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mds": "1/5", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mds_balancer": "1/5", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mds_locker": "1/5", 2024-04-26T04:32:19.550 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mds_log": "1/5", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mds_log_expire": "1/5", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mds_migrator": "1/5", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mds_quiesce": "3/5", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_memstore": "1/5", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mgr": "2/5", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mgrc": "1/5", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_mon": "20/20", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_monc": "0/10", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_ms": "1/1", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_none": "0/5", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_objclass": "0/5", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_objectcacher": "0/5", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_objecter": "0/1", 2024-04-26T04:32:19.551 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_optracker": "0/5", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_osd": "1/5", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_paxos": "20/20", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_perfcounter": "1/5", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_prioritycache": "1/5", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rados": "0/5", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rbd": "0/5", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rbd_mirror": "0/5", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rbd_pwl": "0/5", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rbd_replay": "0/5", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_refs": "0/0", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_reserver": "1/1", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rgw": "1/5", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rgw_access": "1/5", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rgw_datacache": "1/5", 2024-04-26T04:32:19.552 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rgw_dbstore": "1/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rgw_flight": "1/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rgw_lifecycle": "1/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rgw_sync": "1/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_rocksdb": "4/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore": "0/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_backref": "0/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_cache": "0/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_cleaner": "0/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_device": "0/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_epm": "0/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_journal": "0/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_lba": "0/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_odata": "0/5", 2024-04-26T04:32:19.553 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_omap": "0/5", 2024-04-26T04:32:19.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_onode": "0/5", 2024-04-26T04:32:19.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_t": "0/5", 2024-04-26T04:32:19.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_seastore_tm": "0/5", 2024-04-26T04:32:19.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_striper": "0/1", 2024-04-26T04:32:19.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_test": "0/5", 2024-04-26T04:32:19.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_throttle": "1/1", 2024-04-26T04:32:19.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_timer": "0/1", 2024-04-26T04:32:19.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_tp": "0/5", 2024-04-26T04:32:19.554 INFO:teuthology.orchestra.run.smithi012.stdout: "debug_trace": "1/5", 2024-04-26T04:32:19.554 INFO:teuthology.orchestra.run.smithi012.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-04-26T04:32:19.554 INFO:teuthology.orchestra.run.smithi012.stdout: "device_failure_prediction_mode": "none", 2024-04-26T04:32:19.554 INFO:teuthology.orchestra.run.smithi012.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-04-26T04:32:19.554 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "err_to_graylog": "false", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "err_to_journald": "false", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "err_to_stderr": "true", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "err_to_syslog": "false", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "event_tracing": "false", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "exporter_addr": "0.0.0.0", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "exporter_http_port": "9926", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "exporter_prio_limit": "5", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "exporter_sort_metrics": "true", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "exporter_stats_period": "5", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "fake_statfs_for_testing": "0", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "fatal_signal_handlers": "true", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "filer_max_purge_ops": "10", 2024-04-26T04:32:19.555 INFO:teuthology.orchestra.run.smithi012.stdout: "filer_max_truncate_ops": "128", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_apply_finisher_threads": "1", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_blackhole": "false", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_btrfs_clone_range": "true", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_btrfs_snap": "true", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_caller_concurrency": "10", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_collect_device_partition_information": "true", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_commit_timeout": "600.000000", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_debug_inject_read_err": "false", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_debug_omap_check": "false", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_debug_random_read_err": "0.000000", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_debug_verify_split": "false", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_dump_file": "", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-04-26T04:32:19.556 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fadvise": "true", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fail_eio": "true", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fd_cache_shards": "16", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fd_cache_size": "128", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fiemap": "false", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fiemap_threshold": "4096", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_index_retry_probability": "0.000000", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_inject_stall": "0", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_journal_parallel": "false", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_journal_trailing": "false", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_journal_writeahead": "false", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_kill_at": "0", 2024-04-26T04:32:19.557 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattr_size": "0", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattrs": "0", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattrs_other": "2", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_sync_interval": "5.000000", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_xattr_value_size": "0", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-04-26T04:32:19.558 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_merge_threshold": "-10", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_min_sync_interval": "0.010000", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_odsync_write": "false", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_omap_backend": "rocksdb", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_omap_backend_path": "", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_omap_header_cache_size": "1024", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_ondisk_finisher_threads": "1", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_op_thread_timeout": "60", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_op_threads": "2", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_punch_hole": "false", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-04-26T04:32:19.559 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_max_bytes": "104857600", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_queue_max_ops": "50", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_seek_data_hole": "false", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_sloppy_crc": "false", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_splice": "false", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_split_multiple": "2", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_split_rand_factor": "20", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_update_to": "1000", 2024-04-26T04:32:19.560 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_enable": "true", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_xfs_extsize": "false", 2024-04-26T04:32:19.561 INFO:teuthology.orchestra.run.smithi012.stdout: "filestore_zfs_snap": "false", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fio_dir": "/tmp/fio", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fsid": "12c27fbe-0161-48d0-88ec-42a226685437", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_allow_other": "true", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_atomic_o_trunc": "true", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_big_writes": "true", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_debug": "false", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_default_permissions": "false", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_disable_pagecache": "false", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_max_write": "0", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_multithreaded": "true", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_require_active_mds": "true", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_set_user_groups": "true", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_splice_move": "true", 2024-04-26T04:32:19.562 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_splice_read": "true", 2024-04-26T04:32:19.563 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_splice_write": "true", 2024-04-26T04:32:19.563 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_syncfs_on_mksnap": "true", 2024-04-26T04:32:19.563 INFO:teuthology.orchestra.run.smithi012.stdout: "fuse_use_invalidate_cb": "true", 2024-04-26T04:32:19.563 INFO:teuthology.orchestra.run.smithi012.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-04-26T04:32:19.563 INFO:teuthology.orchestra.run.smithi012.stdout: "gss_target_name": "ceph", 2024-04-26T04:32:19.563 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_file": "", 2024-04-26T04:32:19.563 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_inject_failure": "0", 2024-04-26T04:32:19.563 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_interval": "5", 2024-04-26T04:32:19.564 INFO:teuthology.orchestra.run.smithi012.stdout: "host": "smithi012", 2024-04-26T04:32:19.564 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-04-26T04:32:19.564 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-04-26T04:32:19.564 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_max_size": "1073741824", 2024-04-26T04:32:19.564 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-04-26T04:32:19.564 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-04-26T04:32:19.564 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-04-26T04:32:19.564 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-04-26T04:32:19.565 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-04-26T04:32:19.565 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-04-26T04:32:19.565 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-04-26T04:32:19.565 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-04-26T04:32:19.565 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-04-26T04:32:19.565 INFO:teuthology.orchestra.run.smithi012.stdout: "immutable_object_cache_watermark": "0.900000", 2024-04-26T04:32:19.565 INFO:teuthology.orchestra.run.smithi012.stdout: "inject_early_sigterm": "false", 2024-04-26T04:32:19.565 INFO:teuthology.orchestra.run.smithi012.stdout: "jaeger_agent_port": "6799", 2024-04-26T04:32:19.565 INFO:teuthology.orchestra.run.smithi012.stdout: "jaeger_tracing_enable": "false", 2024-04-26T04:32:19.565 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_aio": "true", 2024-04-26T04:32:19.565 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_align_min_size": "65536", 2024-04-26T04:32:19.565 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_block_align": "true", 2024-04-26T04:32:19.565 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_block_size": "4096", 2024-04-26T04:32:19.566 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_dio": "true", 2024-04-26T04:32:19.566 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_discard": "false", 2024-04-26T04:32:19.566 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_force_aio": "false", 2024-04-26T04:32:19.566 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_ignore_corruption": "false", 2024-04-26T04:32:19.566 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_max_write_bytes": "10485760", 2024-04-26T04:32:19.566 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_max_write_entries": "100", 2024-04-26T04:32:19.566 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_replay_from": "0", 2024-04-26T04:32:19.566 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_throttle_high_multiple": "0.000000", 2024-04-26T04:32:19.566 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-04-26T04:32:19.566 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-04-26T04:32:19.566 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_throttle_max_multiple": "0.000000", 2024-04-26T04:32:19.566 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_write_header_frequency": "0", 2024-04-26T04:32:19.566 INFO:teuthology.orchestra.run.smithi012.stdout: "journal_zero_on_create": "false", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "journaler_prefetch_periods": "10", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "journaler_prezero_periods": "5", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "journaler_write_head_interval": "15", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "key": "", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "keyfile": "", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_backend": "rocksdb", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_default_stripe_size": "65536", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_fsck_on_mount": "false", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_fsck_on_mount_deep": "true", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_max_bytes": "67108864", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_max_ops": "512", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_nid_prealloc": "1024", 2024-04-26T04:32:19.567 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_onode_map_size": "1024", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_sync_submit_transaction": "false", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "kstore_sync_transaction": "false", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "librados_thread_count": "2", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "lockdep": "false", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "lockdep_force_backtrace": "false", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "log_coarse_timestamps": "true", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "log_flush_on_exit": "false", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "log_graylog_host": "127.0.0.1", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "log_graylog_port": "12201", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "log_max_new": "1000", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "log_max_recent": "10000", 2024-04-26T04:32:19.568 INFO:teuthology.orchestra.run.smithi012.stdout: "log_stderr_prefix": "", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "log_stop_at_utilization": "0.970000", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_file": "true", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_graylog": "false", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_journald": "false", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_stderr": "false", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "log_to_syslog": "false", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "max_rotating_auth_attempts": "10", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_action_on_write_error": "1", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_alternate_name_max": "8192", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_asio_thread_count": "2", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_export_pin": "true", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_fragment_dirs": "true", 2024-04-26T04:32:19.569 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_fragment_interval": "5", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_fragment_size_max": "100000", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_idle_threshold": "0.000000", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_interval": "10", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_max": "-1", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_max_until": "-1", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_merge_size": "50", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_midchunk": "0.300000", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_min_rebalance": "0.100000", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_min_start": "0.200000", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_minchunk": "0.001000", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_mode": "0", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_need_max": "1.200000", 2024-04-26T04:32:19.570 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_need_min": "0.800000", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_overload_epochs": "2", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_sample_interval": "3.000000", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_split_bits": "3", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_split_rd": "25000.000000", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_split_size": "10000", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_split_wr": "10000.000000", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_target_decay": "10.000000", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_beacon_grace": "15.000000", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_beacon_interval": "4.000000", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_beacon_mon_down_grace": "60", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_memory_limit": "4294967296", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_mid": "0.700000", 2024-04-26T04:32:19.571 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_quiesce_delay": "0", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_quiesce_sleep": "200", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_quiesce_splitauth": "true", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_quiesce_threshold": "524288", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_release_free_interval": "10", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_reservation": "0.050000", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_trim_interval": "1", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cache_trim_threshold": "262144", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_client_delegate_inos_pct": "50", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_client_prealloc_inos": "1000", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-04-26T04:32:19.572 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_connect_bootstrapping": "false", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_damage_table_max_entries": "10000", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_debug_auth_pins": "false", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_debug_frag": "false", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_debug_scatterstat": "false", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_debug_subtrees": "false", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_decay_halflife": "5.000000", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_default_dir_hash": "2", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_defer_session_stale": "true", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_deny_all_reconnect": "false", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dir_keys_per_op": "16384", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dir_max_commit_size": "10", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dir_max_entries": "0", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dir_prefetch": "true", 2024-04-26T04:32:19.573 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dirstat_min_interval": "1.000000", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dump_cache_after_rejoin": "false", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dump_cache_on_map": "false", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dump_cache_threshold_file": "0", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_early_reply": "true", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_enable_op_tracker": "true", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_enforce_unique_name": "true", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_export_ephemeral_distributed": "true", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_export_ephemeral_random": "true", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_forward_all_requests_to_auth": "false", 2024-04-26T04:32:19.574 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_health_cache_threshold": "1.500000", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_health_summarize_threshold": "10", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_heartbeat_grace": "15.000000", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_heartbeat_reset_grace": "1000", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_inject_health_dummy": "false", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_inject_migrator_session_race": "false", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_join_fs": "", 2024-04-26T04:32:19.575 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_journal_format": "1", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_create_at": "0", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_export_at": "0", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_import_at": "0", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_journal_at": "0", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_journal_expire_at": "0", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_journal_replay_at": "0", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_link_at": "0", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_mdstable_at": "0", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_openc_at": "0", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_rename_at": "0", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_shutdown_at": "0", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_event_large_threshold": "524288", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_events_per_segment": "1024", 2024-04-26T04:32:19.576 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_major_segment_event_ratio": "12", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_max_events": "-1", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_max_segments": "128", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_pause": "false", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_segment_size": "0", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_skip_corrupt_events": "false", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_skip_unbounded_events": "false", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_trim_threshold": "128", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_log_warn_factor": "2.000000", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_caps_per_client": "1048576", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_completed_flushes": "100000", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_completed_requests": "100000", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_export_size": "20971520", 2024-04-26T04:32:19.577 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_file_recover": "32", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_purge_files": "64", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_purge_ops": "8192", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_max_snaps_per_dir": "100", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_metrics_update_interval": "2", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_min_caps_per_client": "100", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_min_caps_working_set": "10000", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_numa_node": "-1", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_op_complaint_time": "30.000000", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_op_history_duration": "600", 2024-04-26T04:32:19.578 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_op_history_size": "20", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_op_history_slow_op_size": "20", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_op_history_slow_op_threshold": "10", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_op_log_threshold": "5", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_ping_grace": "15", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_ping_interval": "5", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_recall_max_caps": "30000", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_recall_max_decay_threshold": "131072", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_recall_warning_threshold": "262144", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_reconnect_timeout": "45.000000", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_replay_interval": "1.000000", 2024-04-26T04:32:19.579 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_root_ino_gid": "0", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_root_ino_uid": "0", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_blocklist_on_evict": "true", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_blocklist_on_timeout": "true", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_session_metadata_threshold": "16777216", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_shutdown_check": "0", 2024-04-26T04:32:19.580 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_skip_ino": "0", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_sleep_rank_change": "0.000000", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_snap_max_uid": "4294967294", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_snap_min_uid": "0", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_snap_rstat": "false", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_standby_replay_damaged": "false", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_symlink_recovery": "true", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_task_status_update_interval": "2.000000", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_thrash_exports": "0", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_thrash_fragments": "0", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_tick_interval": "5.000000", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_valgrind_exit": "false", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_verify_backtrace": "1", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_verify_scatter": "false", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_wipe_ino_prealloc": "false", 2024-04-26T04:32:19.581 INFO:teuthology.orchestra.run.smithi012.stdout: "mds_wipe_sessions": "false", 2024-04-26T04:32:19.582 INFO:teuthology.orchestra.run.smithi012.stdout: "mempool_debug": "false", 2024-04-26T04:32:19.582 INFO:teuthology.orchestra.run.smithi012.stdout: "memstore_debug_omit_block_device_write": "false", 2024-04-26T04:32:19.582 INFO:teuthology.orchestra.run.smithi012.stdout: "memstore_device_bytes": "1073741824", 2024-04-26T04:32:19.582 INFO:teuthology.orchestra.run.smithi012.stdout: "memstore_page_set": "false", 2024-04-26T04:32:19.582 INFO:teuthology.orchestra.run.smithi012.stdout: "memstore_page_size": "65536", 2024-04-26T04:32:19.582 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_client_bytes": "134217728", 2024-04-26T04:32:19.582 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_client_messages": "512", 2024-04-26T04:32:19.582 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-04-26T04:32:19.582 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_connect_retry_interval": "1.000000", 2024-04-26T04:32:19.582 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-04-26T04:32:19.582 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-04-26T04:32:19.582 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_disabled_modules": "", 2024-04-26T04:32:19.582 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_max_pg_num_change": "128", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_mds_bytes": "134217728", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_mds_messages": "128", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_mon_bytes": "134217728", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_mon_messages": "128", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_osd_bytes": "536870912", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_osd_messages": "8192", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_pool": "true", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_service_beacon_grace": "60.000000", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_standby_modules": "true", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_stats_period": "5", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_stats_threshold": "5", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_tick_period": "2", 2024-04-26T04:32:19.583 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_accept_timeout_factor": "2.000000", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_allow_pool_delete": "true", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_allow_pool_size_one": "true", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_auth_validate_all_caps": "true", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_bytes": "104857600", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_directed_command_retry": "2", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_hunt_interval": "3.000000", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_hunt_parallel": "3", 2024-04-26T04:32:19.584 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_log_interval": "1.000000", 2024-04-26T04:32:19.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-04-26T04:32:19.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_ping_interval": "10.000000", 2024-04-26T04:32:19.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_ping_timeout": "30.000000", 2024-04-26T04:32:19.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_client_target_rank": "-1", 2024-04-26T04:32:19.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_clock_drift_allowed": "1.000000", 2024-04-26T04:32:19.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-04-26T04:32:19.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-04-26T04:32:19.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_file_level": "debug", 2024-04-26T04:32:19.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_file": "true", 2024-04-26T04:32:19.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_graylog": "false", 2024-04-26T04:32:19.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-04-26T04:32:19.585 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_journald": "false", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_stderr": "false", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_compact_on_bootstrap": "false", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_compact_on_start": "false", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_compact_on_trim": "true", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_con_tracker_persist_interval": "10", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_con_tracker_score_halflife": "43200", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_config_key_max_entry_size": "65536", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_cpu_threads": "4", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_crush_min_required_version": "hammer", 2024-04-26T04:32:19.586 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_daemon_bytes": "419430400", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_data_avail_crit": "5", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_data_avail_warn": "5", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_data_size_warn": "16106127360", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_block_osdmap_trim": "false", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_dump_json": "false", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_dump_transactions": "false", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_extra_checks": "false", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_no_require_reef": "false", 2024-04-26T04:32:19.587 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_no_require_squid": "false", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_delta_reset_interval": "10.000000", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_dns_srv_name": "ceph-mon", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_down_mkfs_grace": "120", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_down_uptime_grace": "60", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_election_default_strategy": "3", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_election_timeout": "5.000000", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_elector_ping_divisor": "2", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_elector_ping_timeout": "2.000000", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_enable_op_tracker": "true", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_fake_pool_delete": "false", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_force_quorum_join": "false", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_globalid_prealloc": "10000", 2024-04-26T04:32:19.588 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_health_detail_to_clog": "false", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_health_log_update_period": "5", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_health_max_detail": "50", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_health_to_clog": "true", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_health_to_clog_interval": "600", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_host": "172.21.15.12,172.21.15.113,172.21.15.137", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_host_override": "", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_initial_members": "", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_keyvaluedb": "rocksdb", 2024-04-26T04:32:19.589 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_lease": "5.000000", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_log_full_interval": "50", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_log_max": "10000", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_log_max_summary": "50", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_log_entries_per_event": "4096", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_log_epochs": "500", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_mdsmap_epochs": "500", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_mgrmap_epochs": "500", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_osd": "10000", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_pg_per_osd": "10000", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_pool_pg_num": "65536", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-04-26T04:32:19.590 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mds_force_trim_to": "0", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mds_skip_sanity": "false", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_memory_autotune": "true", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_memory_target": "2147483648", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mgr_beacon_grace": "30", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mgr_digest_period": "5", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mgr_inactive_grace": "60", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mgr_mkfs_grace": "240", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_min_osdmap_epochs": "500", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_op_complaint_time": "30", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_op_history_duration": "600", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_op_history_size": "20", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_op_history_slow_op_size": "20", 2024-04-26T04:32:19.591 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_op_history_slow_op_threshold": "10", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_op_log_threshold": "5", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_auto_mark_in": "false", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_auto_mark_new_in": "true", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_cache_size": "500", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_cache_size_min": "134217728", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_crush_smoke_test": "true", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_destroyed_out_interval": "600", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_down_out_interval": "600", 2024-04-26T04:32:19.592 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_force_trim_to": "0", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_full_ratio": "0.950000", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_laggy_halflife": "3600", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_laggy_max_interval": "300", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_laggy_weight": "0.300000", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_max_creating_pgs": "1024", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_max_initial_pgs": "1024", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_min_down_reporters": "2", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-04-26T04:32:19.593 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_prime_pg_temp": "true", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_report_timeout": "900", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_warn_num_repaired": "10", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osd_warn_op_age": "32.000000", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osdmap_full_prune_interval": "10", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osdmap_full_prune_min": "10000", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pg_stuck_threshold": "60", 2024-04-26T04:32:19.594 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pg_warn_min_objects": "10000", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pg_warn_min_per_osd": "0", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pool_quota_crit_threshold": "0", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_pool_quota_warn_threshold": "0", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_probe_timeout": "2.000000", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_reweight_max_change": "0.050000", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_reweight_max_osds": "4", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-04-26T04:32:19.595 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_scrub_interval": "300", 2024-04-26T04:32:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_scrub_max_keys": "100", 2024-04-26T04:32:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_scrub_timeout": "300", 2024-04-26T04:32:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_session_timeout": "300", 2024-04-26T04:32:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_smart_report_timeout": "5", 2024-04-26T04:32:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_stat_smooth_intervals": "6", 2024-04-26T04:32:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-04-26T04:32:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_stretch_pool_min_size": "2", 2024-04-26T04:32:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_stretch_pool_size": "4", 2024-04-26T04:32:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-04-26T04:32:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_subscribe_interval": "86400.000000", 2024-04-26T04:32:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_sync_debug": "false", 2024-04-26T04:32:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_sync_max_payload_keys": "2000", 2024-04-26T04:32:19.596 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_sync_max_payload_size": "1048576", 2024-04-26T04:32:19.597 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_sync_provider_kill_at": "0", 2024-04-26T04:32:19.597 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_sync_requester_kill_at": "0", 2024-04-26T04:32:19.597 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_sync_timeout": "60.000000", 2024-04-26T04:32:19.597 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_target_pg_per_osd": "100", 2024-04-26T04:32:19.597 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_tick_interval": "5", 2024-04-26T04:32:19.597 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_timecheck_interval": "300.000000", 2024-04-26T04:32:19.597 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-04-26T04:32:19.597 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_use_min_delay_socket": "false", 2024-04-26T04:32:19.597 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_older_version_delay": "604800", 2024-04-26T04:32:19.597 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-04-26T04:32:19.597 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-04-26T04:32:19.597 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-04-26T04:32:19.597 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_filestore_osds": "false", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_misplaced": "false", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_older_version": "true", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_pool_no_app": "true", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_on_too_few_osds": "false", 2024-04-26T04:32:19.598 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "monmap": "", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "motr_tracing_enabled": "false", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_op_threads": "3", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_buffer_size": "131072", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_cm": "false", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_device_name": "", 2024-04-26T04:32:19.599 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_dscp": "96", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_gid_idx": "0", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_local_gid": "", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_polling_us": "1000", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_port_num": "1", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_roce_ver": "1", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_send_buffers": "1024", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_sl": "3", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_support_srq": "true", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_rdma_type": "ib", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_async_reap_threshold": "5", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_before_connect": "false", 2024-04-26T04:32:19.600 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_ipv4": "true", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_ipv6": "false", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_msgr1": "true", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_msgr2": "true", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_port_max": "7568", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_port_min": "6800", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_prefer_ipv4": "false", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_retry_count": "3", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_bind_retry_delay": "5", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_blackhole_client": "false", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_blackhole_mds": "false", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_blackhole_mgr": "false", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_blackhole_mon": "false", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_blackhole_osd": "false", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_client_mode": "crc secure", 2024-04-26T04:32:19.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_cluster_mode": "crc secure", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_cluster_type": "", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_compress_secure": "false", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_connection_idle_timeout": "900", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_connection_ready_timeout": "10", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_crc_data": "true", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_crc_header": "true", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_die_on_bad_msg": "false", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_die_on_bug": "true", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_die_on_old_message": "true", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_die_on_skipped_message": "false", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_die_on_unhandled_msg": "false", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_coremask": "0xF", 2024-04-26T04:32:19.602 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_devs_allowlist": "", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_enable_tso": "true", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_hugepages": "", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_hw_flow_control": "true", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_lro": "true", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_memory_channel": "4", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_pmd": "", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_port_id": "0", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-04-26T04:32:19.603 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dump_corrupt_message_level": "1", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_dump_on_send": "false", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_initial_backoff": "0.200000", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_inject_delay_max": "1.000000", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_inject_delay_probability": "0.000000", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_inject_delay_type": "", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_inject_internal_delays": "0.000000", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_inject_network_congestion": "0", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_inject_socket_failures": "0", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_learn_addr_from_peer": "true", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_max_accept_failures": "4", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_max_backoff": "15.000000", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_mon_client_mode": "secure crc", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_mon_cluster_mode": "secure crc", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_mon_service_mode": "secure crc", 2024-04-26T04:32:19.604 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_osd_compress_min_size": "1024", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_osd_compress_mode": "none", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_osd_compression_algorithm": "snappy", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_pq_min_cost": "65536", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_public_type": "", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_service_mode": "crc secure", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_tcp_listen_backlog": "512", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_tcp_nodelay": "true", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_tcp_rcvbuf": "0", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "ms_type": "async+posix", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "no_config_file": "false", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_completion_locks_per_session": "32", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_debug_inject_relock_delay": "false", 2024-04-26T04:32:19.605 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_inflight_op_bytes": "104857600", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_inflight_ops": "1024", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_inject_no_watch_ping": "false", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_tick_interval": "5.000000", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "objecter_timeout": "10.000000", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "objectstore_blackhole": "false", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "openssl_engine_opts": "", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_delay_time": "5.000000", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_hist_halflife": "1000", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_max_low_ops": "2", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_max_ops": "4", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_quantize_effort": "0.100000", 2024-04-26T04:32:19.606 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_agent_slop": "0.020000", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_allow_recovery_below_min_size": "true", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_asio_thread_count": "2", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_async_recovery_min_cost": "1", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_auto_mark_unfound_lost": "false", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_backfill_retry_interval": "30.000000", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_backfill_scan_max": "512", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_backfill_scan_min": "64", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_backoff_on_degraded": "false", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_backoff_on_peering": "false", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_backoff_on_unfound": "true", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_beacon_report_interval": "300", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bench_duration": "30", 2024-04-26T04:32:19.607 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bench_max_block_size": "67108864", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_bench_small_size_max_iops": "100", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_blkin_trace_all": "false", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_blocked_scrub_grace_period": "120", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_check_for_log_corruption": "false", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_class_update_on_start": "true", 2024-04-26T04:32:19.608 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_client_message_cap": "256", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_client_message_size_cap": "524288000", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_client_op_priority": "63", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_client_watch_timeout": "30", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_command_max_records": "256", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_compact_on_start": "false", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_crush_chooseleaf_type": "0", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_crush_initial_weight": "-1.000000", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_crush_update_on_start": "true", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_crush_update_weight_set": "true", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-04-26T04:32:19.609 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_drop_ping_duration": "0", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_feed_pullee": "-1", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_inject_dispatch_delay_probability": "0.100000", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_misdirected_ops": "false", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_no_acting_change": "false", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_no_purge_strays": "false", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_op_order": "false", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_pg_log_writeout": "false", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_pretend_recovery_active": "false", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-04-26T04:32:19.610 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_shutdown": "false", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_trim_objects": "false", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_verify_cached_snaps": "false", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_verify_missing_on_start": "false", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_verify_snaps": "false", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_keys": "1024", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_stride": "524288", 2024-04-26T04:32:19.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_default_data_pool_replay_window": "5", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_default_notify_timeout": "30", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_delete_sleep": "0.000000", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_discard_disconnected_ops": "true", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_enable_op_tracker": "true", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_extblkdev_plugins": "vdo", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_fast_info": "true", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_fast_shutdown": "true", 2024-04-26T04:32:19.612 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_fast_shutdown_timeout": "15", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_function_tracing": "false", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_grace": "20", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_interval": "6", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_min_peers": "10", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_min_size": "2000", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_stale": "600", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_hit_set_max_size": "100000", 2024-04-26T04:32:19.613 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_hit_set_min_size": "1000", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_journal_flush_on_shutdown": "true", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_journal_size": "5120", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_kill_backfill_at": "0", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_loop_before_reset_tphandle": "64", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_map_cache_size": "50", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_map_dedup": "true", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_map_message_max": "40", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_map_message_max_bytes": "10485760", 2024-04-26T04:32:19.614 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_map_share_max_epochs": "40", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_attr_name_len": "100", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_attr_size": "0", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_backfills": "1", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_markdown_count": "5", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_markdown_period": "600", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_object_name_len": "2048", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_object_namespace_len": "256", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_object_size": "134217728", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_omap_entries_per_request": "1024", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_pg_blocked_by": "16", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_pg_log_entries": "10000", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-04-26T04:32:19.615 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_pgls": "1024", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_push_cost": "8388608", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_push_objects": "10", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_scrubs": "3", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_trimming_pgs": "2", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_write_op_reply_len": "64", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_max_write_size": "90", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-04-26T04:32:19.616 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_override_recovery_settings": "false", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_profile": "balanced", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mclock_skip_benchmark": "false", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_base": "805306368", 2024-04-26T04:32:19.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_cache_min": "134217728", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_target": "4294967296", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_target_autotune": "false", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_min_pg_log_entries": "250", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_min_recovery_priority": "0", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mon_heartbeat_interval": "30", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mon_report_interval": "5", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mon_report_max_in_flight": "2", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_num_cache_shards": "32", 2024-04-26T04:32:19.618 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_num_op_tracker_shard": "32", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_numa_auto_affinity": "true", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_numa_node": "-1", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_numa_prefer_iface": "true", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_object_clean_region_max_num_intervals": "1000", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_objecter_finishers": "1", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_objectstore": "bluestore", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_objectstore_fuse": "false", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_objectstore_tracing": "false", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_complaint_time": "30.000000", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_history_duration": "600", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_history_size": "20", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_history_slow_op_size": "20", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-04-26T04:32:19.619 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_log_threshold": "5", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_num_shards": "0", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_num_shards_hdd": "5", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_num_shards_ssd": "8", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_num_threads_per_shard": "0", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_pq_min_cost": "65536", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_queue": "mclock_scheduler", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_queue_cut_off": "high", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_thread_suicide_timeout": "150", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_op_thread_timeout": "15", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_open_classes_on_start": "true", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_os_flags": "0", 2024-04-26T04:32:19.620 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_peering_op_priority": "255", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_delete_cost": "1048576", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_delete_priority": "5", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_log_dups_tracked": "3000", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_log_trim_max": "10000", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_log_trim_min": "100", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_object_context_cache_count": "64", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-04-26T04:32:19.621 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_crimson": "false", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_crush_rule": "-1", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_ec_fast_read": "false", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_flag_bulk": "false", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_flag_nodelete": "false", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_flags": "0", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-04-26T04:32:19.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_min_size": "0", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_pg_num": "32", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_pgp_num": "0", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_read_ratio": "70", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_size": "2", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_default_type": "replicated", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_pool_use_gmt_hitset": "true", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_push_per_object_cost": "1000", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_read_ec_check_for_errors": "false", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recover_clone_overlap": "true", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recover_clone_overlap_limit": "10", 2024-04-26T04:32:19.623 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_cost": "20971520", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_delay_start": "0.000000", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_max_active": "0", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_max_active_hdd": "3", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_max_active_ssd": "10", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_max_chunk": "8388608", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_max_single_start": "1", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_op_priority": "3", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_op_warn_multiple": "16", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_priority": "5", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_retry_interval": "30.000000", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_sleep": "0.000000", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-04-26T04:32:19.624 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_repair_during_recovery": "false", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_requested_scrub_priority": "120", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_rollback_to_cluster_snap": "", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_auto_repair": "false", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_begin_hour": "0", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_begin_week_day": "0", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_chunk_max": "25", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_chunk_min": "5", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_cost": "52428800", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_disable_reservation_queuing": "true", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_during_recovery": "false", 2024-04-26T04:32:19.625 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_end_hour": "0", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_end_week_day": "0", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_event_cost": "4096", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_invalid_stats": "true", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_load_threshold": "0.500000", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_max_interval": "604800.000000", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_max_preemptions": "5", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_min_interval": "86400.000000", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_priority": "5", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_scrub_sleep": "0.000000", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-04-26T04:32:19.626 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_shutdown_pgref_assert": "false", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_skip_data_digest": "false", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_smart_report_timeout": "5", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_snap_trim_cost": "1048576", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_snap_trim_priority": "5", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_snap_trim_sleep": "0.000000", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_stats_update_period_scrubbing": "15", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_target_transaction_size": "30", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-04-26T04:32:19.627 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_default_cache_mode": "writeback", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_tracing": "false", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_use_stale_snap": "false", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "osdc_blkin_trace_all": "false", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_kill_at": "0", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_max_join_drift": "10", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_min": "500", 2024-04-26T04:32:19.628 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_min_wait": "0.050000", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_propose_interval": "1.000000", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_service_trim_max": "500", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_service_trim_max_multiplier": "20", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_service_trim_min": "250", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_stash_full_interval": "25", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_trim_max": "500", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "paxos_trim_min": "250", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "perf": "true", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin_dir": "/usr/lib/ceph", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "-", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrv": "", 2024-04-26T04:32:19.629 INFO:teuthology.orchestra.run.smithi012.stdout: "public_bind_addr": "-", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "public_network": "", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "public_network_interface": "", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "qat_compressor_enabled": "false", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "qat_compressor_session_max_number": "256", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rados_mon_op_timeout": "0", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rados_osd_op_timeout": "0", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rados_tracing": "false", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_atime_update_interval": "60", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_balance_parent_reads": "false", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_balance_snap_reads": "false", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_blkin_trace_all": "false", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_blocklist_expire_seconds": "0", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_blocklist_on_break_lock": "true", 2024-04-26T04:32:19.630 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache": "true", 2024-04-26T04:32:19.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_block_writes_upfront": "false", 2024-04-26T04:32:19.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_max_dirty": "25165824", 2024-04-26T04:32:19.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-04-26T04:32:19.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_max_dirty_object": "0", 2024-04-26T04:32:19.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_policy": "writearound", 2024-04-26T04:32:19.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_size": "33554432", 2024-04-26T04:32:19.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_target_dirty": "16777216", 2024-04-26T04:32:19.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-04-26T04:32:19.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_clone_copy_on_read": "false", 2024-04-26T04:32:19.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_compression_hint": "none", 2024-04-26T04:32:19.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_concurrent_management_ops": "10", 2024-04-26T04:32:19.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-04-26T04:32:19.631 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_clone_format": "auto", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_data_pool": "", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_features": "61", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_format": "2", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_map_options": "", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_order": "22", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_pool": "rbd", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_stripe_count": "0", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_default_stripe_unit": "0", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_disable_zero_copy_writes": "true", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_discard_granularity_bytes": "65536", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_enable_alloc_hint": "true", 2024-04-26T04:32:19.632 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_io_scheduler": "simple", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_commit_age": "5.000000", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_object_flush_interval": "0", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_order": "24", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_pool": "", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_journal_splay_width": "4", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_localize_parent_reads": "false", 2024-04-26T04:32:19.633 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_localize_snap_reads": "false", 2024-04-26T04:32:19.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-04-26T04:32:19.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-04-26T04:32:19.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-04-26T04:32:19.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-04-26T04:32:19.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-04-26T04:32:19.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-04-26T04:32:19.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_image_policy_type": "simple", 2024-04-26T04:32:19.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-04-26T04:32:19.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-04-26T04:32:19.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-04-26T04:32:19.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-04-26T04:32:19.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-04-26T04:32:19.634 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_autotune": "true", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_base": "805306368", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_memory_target": "4294967296", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirroring_delete_delay": "0", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirroring_replay_delay": "0", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-04-26T04:32:19.635 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-04-26T04:32:19.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_move_to_trash_on_remove": "false", 2024-04-26T04:32:19.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-04-26T04:32:19.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_mtime_update_interval": "60", 2024-04-26T04:32:19.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_non_blocking_aio": "true", 2024-04-26T04:32:19.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_op_thread_timeout": "60", 2024-04-26T04:32:19.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_op_threads": "1", 2024-04-26T04:32:19.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_parent_cache_enabled": "false", 2024-04-26T04:32:19.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_persistent_cache_mode": "disabled", 2024-04-26T04:32:19.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_persistent_cache_path": "/tmp", 2024-04-26T04:32:19.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_persistent_cache_size": "1073741824", 2024-04-26T04:32:19.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_plugins": "", 2024-04-26T04:32:19.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_bps_burst": "0", 2024-04-26T04:32:19.636 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_bps_limit": "0", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_exclude_ops": "0", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_iops_burst": "0", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_iops_limit": "0", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_read_bps_burst": "0", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_read_bps_limit": "0", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_read_iops_burst": "0", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_read_iops_limit": "0", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_schedule_tick_min": "50", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_write_bps_burst": "0", 2024-04-26T04:32:19.637 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_write_bps_limit": "0", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_write_iops_burst": "0", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_qos_write_iops_limit": "0", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_quiesce_notification_attempts": "10", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_read_from_replica_policy": "default", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_readahead_max_bytes": "524288", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_readahead_trigger_requests": "10", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_request_timed_out_seconds": "30", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_skip_partial_discard": "true", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_tracing": "false", 2024-04-26T04:32:19.638 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_validate_names": "true", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd_validate_pool": "true", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "restapi_base_url": "", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "restapi_log_level": "", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_acl_grants_max_num": "100", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_admin_entry": "admin", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_backend_store": "rados", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_barbican_url": "", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_beast_enable_async": "true", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_counters_cache": "false", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_index_max_aio": "128", 2024-04-26T04:32:19.639 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_quota_ttl": "600", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_cache_enabled": "true", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_cache_expiry_interval": "900", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_cache_lru_size": "10000", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_config_store": "rados", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_content_length_compat": "false", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_copy_obj_progress": "true", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_cors_rules_max_num": "100", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_cross_domain_policy": "", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_default_encryption_key": "", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_addr": "", 2024-04-26T04:32:19.640 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_ca_path": "", 2024-04-26T04:32:19.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_client_cert": "", 2024-04-26T04:32:19.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_client_key": "", 2024-04-26T04:32:19.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-04-26T04:32:19.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_password": "", 2024-04-26T04:32:19.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-04-26T04:32:19.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_kmip_username": "", 2024-04-26T04:32:19.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_require_ssl": "true", 2024-04-26T04:32:19.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-04-26T04:32:19.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-04-26T04:32:19.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-04-26T04:32:19.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-04-26T04:32:19.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-04-26T04:32:19.641 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-04-26T04:32:19.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-04-26T04:32:19.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-04-26T04:32:19.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-04-26T04:32:19.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-04-26T04:32:19.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-04-26T04:32:19.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-04-26T04:32:19.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-04-26T04:32:19.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-04-26T04:32:19.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_suppress_logs": "true", 2024-04-26T04:32:19.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_addr": "", 2024-04-26T04:32:19.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_auth": "token", 2024-04-26T04:32:19.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_namespace": "", 2024-04-26T04:32:19.642 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_prefix": "", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_token_file": "", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_curl_buffersize": "524288", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_curl_low_speed_limit": "1024", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_curl_low_speed_time": "300", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_curl_tcp_keepalive": "0", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-04-26T04:32:19.643 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_l1_fadvise": "4", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d4n_host": "127.0.0.1", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_d4n_port": "6379", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_log_changes_size": "1000", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_log_num_shards": "128", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_log_window": "30", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_notify_interval_msec": "0", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_sync_poll_interval": "20", 2024-04-26T04:32:19.644 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_data_sync_spawn_window": "20", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_default_data_log_backing": "fifo", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_default_region_info_oid": "default.region", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_defer_to_bucket_acls": "", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-04-26T04:32:19.645 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_data_res": "500.000000", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dns_name": "", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dns_s3website_name": "", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_dynamic_resharding": "true", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_gc_threads": "true", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_lc_threads": "true", 2024-04-26T04:32:19.646 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_ops_log": "false", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_quota_threads": "true", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_static_website": "false", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enable_usage_log": "false", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_enforce_swift_acls": "true", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_exit_timeout_secs": "120", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_expose_bucket": "false", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_extended_http_attrs": "", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_filter": "none", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_frontends": "beast port=7480", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_max_concurrent_io": "10", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_max_deferred": "50", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-04-26T04:32:19.647 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_max_objs": "32", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_max_queue_size": "134213632", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_max_trim_chunk": "16", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_obj_min_wait": "7200", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_processor_max_time": "3600", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_gc_processor_period": "3600", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_get_obj_window_size": "16777216", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_healthcheck_disabling_path": "", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ignore_get_invalid_range": "false", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_init_timeout": "300", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_kafka_connection_idle": "300", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_kafka_message_timeout": "5000", 2024-04-26T04:32:19.648 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_kafka_sleep_timeout": "10", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_domain": "", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_password": "", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_password_path": "", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_project": "", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_tenant": "", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_token": "", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_token_path": "", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_admin_user": "", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_api_version": "2", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_barbican_domain": "", 2024-04-26T04:32:19.649 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_barbican_password": "", 2024-04-26T04:32:19.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_barbican_project": "", 2024-04-26T04:32:19.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_barbican_tenant": "", 2024-04-26T04:32:19.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_barbican_user": "", 2024-04-26T04:32:19.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-04-26T04:32:19.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_implicit_tenants": "false", 2024-04-26T04:32:19.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-04-26T04:32:19.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_service_token_enabled": "false", 2024-04-26T04:32:19.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_token_cache_size": "10000", 2024-04-26T04:32:19.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_url": "", 2024-04-26T04:32:19.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_keystone_verify_ssl": "true", 2024-04-26T04:32:19.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_debug_interval": "-1", 2024-04-26T04:32:19.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_lock_max_time": "90", 2024-04-26T04:32:19.650 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_max_objs": "32", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_max_rules": "1000", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_max_worker": "3", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_max_wp_worker": "3", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lc_thread_delay": "0", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ldap_dnattr": "uid", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ldap_searchfilter": "", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ldap_uri": "", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-04-26T04:32:19.651 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_log_http_headers": "", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_log_nonexistent_bucket": "false", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_log_object_name_utc": "false", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_attr_name_len": "0", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_attr_size": "0", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_attrs_num_in_req": "0", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_chunk_size": "4194304", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_concurrent_requests": "1024", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_dynamic_shards": "1999", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_listing_results": "1000", 2024-04-26T04:32:19.652 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_notify_retries": "10", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_objs_per_shard": "100000", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_put_param_size": "1048576", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_put_size": "5368709120", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_max_slo_entries": "1000", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_md_log_max_shards": "64", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_md_notify_interval_msec": "200", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_meta_sync_poll_interval": "20", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_meta_sync_spawn_window": "20", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_mp_lock_max_time": "600", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_multipart_min_part_size": "5242880", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-04-26T04:32:19.653 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_fhcache_size": "2017", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_lru_lanes": "5", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_max_gc": "300", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_run_gc_threads": "false", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_run_lc_threads": "false", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_run_quota_threads": "false", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_run_sync_thread": "false", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_num_async_rados_threads": "32", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_num_control_oids": "8", 2024-04-26T04:32:19.654 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_numa_node": "-1", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_obj_stripe_size": "4194304", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_objexp_chunk_size": "100", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_objexp_gc_interval": "600", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_objexp_hints_num_shards": "127", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_op_thread_timeout": "600", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_op_tracing": "false", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_opa_token": "", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_opa_url": "", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_opa_verify_ssl": "true", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-04-26T04:32:19.655 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ops_log_rados": "false", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_ops_log_socket_path": "", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_period_push_interval": "2.000000", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_period_push_interval_max": "30.000000", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_period_root_pool": ".rgw.root", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_posix_cache_lanes": "3", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_posix_cache_max_buckets": "100", 2024-04-26T04:32:19.656 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_posix_cache_partitions": "3", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_print_continue": "true", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_print_prohibited_content_length": "false", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_rados_tracing": "false", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_realm": "", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_realm_id": "", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_region": "", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_region_root_pool": ".rgw.root", 2024-04-26T04:32:19.657 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_relaxed_region_enforcement": "false", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_request_uri": "", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_reshard_batch_size": "64", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_reshard_max_aio": "128", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_reshard_num_logs": "16", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_reshard_thread_interval": "600", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_resolve_cname": "false", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_rest_getusage_op_compat": "false", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_run_sync_thread": "true", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-04-26T04:32:19.658 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_auth_use_keystone": "false", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_auth_use_ldap": "false", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_auth_use_rados": "true", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_auth_use_sts": "false", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_s3_success_create_obj_status": "0", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_scheduler_type": "throttler", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_script_uri": "", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_service_provider_name": "", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_client_id": "", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_client_secret": "", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_entry": "sts", 2024-04-26T04:32:19.659 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_key": "sts", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_max_session_duration": "43200", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_min_session_duration": "900", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sts_token_introspection_url": "", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_account_in_url": "false", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_auth_entry": "auth", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_auth_url": "", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_custom_header": "", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_enforce_content_length": "false", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_need_stats": "true", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_tenant_name": "", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_token_expiration": "86400", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_url": "", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_url_prefix": "swift", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_swift_versioning_enabled": "false", 2024-04-26T04:32:19.660 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-04-26T04:32:19.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-04-26T04:32:19.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_lease_period": "120", 2024-04-26T04:32:19.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-04-26T04:32:19.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_log_trim_interval": "1200", 2024-04-26T04:32:19.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-04-26T04:32:19.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-04-26T04:32:19.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-04-26T04:32:19.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_obj_etag_verify": "false", 2024-04-26T04:32:19.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_trace_history_size": "4096", 2024-04-26T04:32:19.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-04-26T04:32:19.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-04-26T04:32:19.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_thread_pool_size": "512", 2024-04-26T04:32:19.661 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_topic_persistency_max_retries": "0", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_topic_require_publish_policy": "false", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_comment": "", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_createby": "", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_encoding": "", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_flag": "false", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_max_size": "5368709120", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_origin": "", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_sha_unit": "524288", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_torrent_tracker": "", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_trust_forwarded_https": "false", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-04-26T04:32:19.662 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_usage_log_tick_interval": "30", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_usage_max_shards": "32", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_usage_max_user_shards": "1", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_use_opa_authz": "false", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_counters_cache": "false", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_counters_cache_size": "10000", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_default_quota_max_size": "-1", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_max_buckets": "1000", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_policies_max_num": "100", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_quota_sync_interval": "86400", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-04-26T04:32:19.663 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_user_unique_email": "true", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_verify_ssl": "true", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_website_routing_rules_max_num": "50", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_zone": "", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_zone_id": "", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_zonegroup": "", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_zonegroup_id": "", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rgwlc_auto_session_clear": "true", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rgwlc_skip_bucket_step": "false", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_block_size": "4096", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-04-26T04:32:19.664 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-04-26T04:32:19.665 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cache_shard_bits": "4", 2024-04-26T04:32:19.665 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cache_size": "536870912", 2024-04-26T04:32:19.665 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cache_type": "binned_lru", 2024-04-26T04:32:19.665 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-04-26T04:32:19.665 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-04-26T04:32:19.665 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-04-26T04:32:19.665 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_collect_compaction_stats": "false", 2024-04-26T04:32:19.665 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_collect_extended_stats": "false", 2024-04-26T04:32:19.665 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_collect_memory_stats": "false", 2024-04-26T04:32:19.665 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-04-26T04:32:19.665 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_index_type": "binary_search", 2024-04-26T04:32:19.665 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_log_to_ceph_log": "true", 2024-04-26T04:32:19.665 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_metadata_block_size": "4096", 2024-04-26T04:32:19.666 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_partition_filters": "false", 2024-04-26T04:32:19.666 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_perf": "false", 2024-04-26T04:32:19.666 INFO:teuthology.orchestra.run.smithi012.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-04-26T04:32:19.666 INFO:teuthology.orchestra.run.smithi012.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-04-26T04:32:19.666 INFO:teuthology.orchestra.run.smithi012.stdout: "rotating_keys_renewal_timeout": "10", 2024-04-26T04:32:19.666 INFO:teuthology.orchestra.run.smithi012.stdout: "run_dir": "/var/run/ceph", 2024-04-26T04:32:19.666 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_block_create": "true", 2024-04-26T04:32:19.666 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_cache_lru_size": "67108864", 2024-04-26T04:32:19.666 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_cbjournal_size": "5368709120", 2024-04-26T04:32:19.666 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_data_delta_based_overwrite": "0", 2024-04-26T04:32:19.666 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_default_max_object_size": "16777216", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_device_size": "53687091200", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_journal_batch_capacity": "16", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_journal_iodepth_limit": "5", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_main_device_type": "SSD", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_max_concurrent_transactions": "8", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "seastore_segment_size": "67108864", 2024-04-26T04:32:19.667 INFO:teuthology.orchestra.run.smithi012.stdout: "set_keepcaps": "false", 2024-04-26T04:32:19.668 INFO:teuthology.orchestra.run.smithi012.stdout: "setgroup": "", 2024-04-26T04:32:19.668 INFO:teuthology.orchestra.run.smithi012.stdout: "setuser": "", 2024-04-26T04:32:19.668 INFO:teuthology.orchestra.run.smithi012.stdout: "setuser_match_path": "", 2024-04-26T04:32:19.668 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_misplaced_ratio": "0.050000", 2024-04-26T04:32:19.668 INFO:teuthology.orchestra.run.smithi012.stdout: "thp": "false", 2024-04-26T04:32:19.668 INFO:teuthology.orchestra.run.smithi012.stdout: "threadpool_default_timeout": "60", 2024-04-26T04:32:19.668 INFO:teuthology.orchestra.run.smithi012.stdout: "threadpool_empty_queue_max_wait": "2", 2024-04-26T04:32:19.668 INFO:teuthology.orchestra.run.smithi012.stdout: "throttler_perf_counter": "true" 2024-04-26T04:32:19.668 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:19.673 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-04-26T04:32:19.821 INFO:teuthology.orchestra.run.smithi012.stdout:mon.a: {} 2024-04-26T04:32:19.822 INFO:teuthology.orchestra.run.smithi012.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-26T04:32:19.834 INFO:teuthology.orchestra.run.smithi012.stdout:mon.b: {} 2024-04-26T04:32:19.834 INFO:teuthology.orchestra.run.smithi012.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-26T04:32:19.847 INFO:teuthology.orchestra.run.smithi012.stdout:mon.c: {} 2024-04-26T04:32:19.847 INFO:teuthology.orchestra.run.smithi012.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-26T04:32:19.863 DEBUG:teuthology.orchestra.run.smithi012:> type ceph-objectstore-tool 2024-04-26T04:32:19.867 INFO:teuthology.orchestra.run.smithi012.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-26T04:32:19.867 DEBUG:teuthology.orchestra.run.smithi178:> type ceph-objectstore-tool 2024-04-26T04:32:19.872 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-26T04:32:19.873 DEBUG:teuthology.orchestra.run.smithi137:> type ceph-objectstore-tool 2024-04-26T04:32:19.878 INFO:teuthology.orchestra.run.smithi137.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-26T04:32:19.879 DEBUG:teuthology.orchestra.run.smithi113:> type ceph-objectstore-tool 2024-04-26T04:32:19.884 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-26T04:32:19.885 INFO:teuthology.run_tasks:Running task rados... 2024-04-26T04:32:19.897 INFO:tasks.rados:Beginning rados... 2024-04-26T04:32:19.897 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-04-26T04:32:19.912 INFO:tasks.rados:joining rados 2024-04-26T04:32:19.912 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-04-26T04:32:19.912 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T04:32:19.912 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T04:32:19.912 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] 2024-04-26T04:32:19.912 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-04-26T04:32:19.913 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-04-26T04:32:19.913 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-04-26T04:32:19.914 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T04:32:19.914 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-04-26T04:32:19.914 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T04:32:19.915 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-04-26T04:32:19.915 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-26T04:32:19.915 INFO:tasks.rados:clients are ['client.0'] 2024-04-26T04:32:19.915 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set shecprofile c=2 crush-failure-domain=osd k=4 m=3 name=shecprofile plugin=shec 2024-04-26T04:32:19.916 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:19.912+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:20.016 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:20.012+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:20.031 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:20.031 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:32:20.031 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-04-26T04:32:20.031 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:20.050 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T04:32:20.116 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:20.115+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:20.136 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:20.137 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:20.171 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:20.171 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:20.176 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:20.176 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:20.195 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T04:32:20.202 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:20.202 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:20.217 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:20.215+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:20.237 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:20.238 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:20.255 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:20.256 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:20.276 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:20.277 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:20.297 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:20.298 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:20.307 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:20.307 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:20.318 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:20.318 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:20.315+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:20.318 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:20.319 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T04:32:20.334 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:20.335 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:20.341 INFO:tasks.ceph.mon.a.smithi012.stderr:2024-04-26T04:32:20.340+0000 7f05ec032640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-04-26T04:32:20.355 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:20.355 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:20.375 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:20.375 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:20.396 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:20.396 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:20.413 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:20.414 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:20.418 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:20.416+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:20.435 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:20.436 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:20.446 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:20.446 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:20.457 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:20.457 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:20.458 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T04:32:20.478 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:20.478 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:20.519 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:20.516+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:20.586 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:20.587 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:20.599 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T04:32:20.646 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:20.644+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:20.678 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-04-26T04:32:20.678 INFO:teuthology.orchestra.run.smithi012.stderr:marked out osd.2. 2024-04-26T04:32:20.685 INFO:teuthology.orchestra.run.smithi012.stderr:load: shec 2024-04-26T04:32:20.714 INFO:tasks.rados:starting run 0 out of 1 2024-04-26T04:32:20.714 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-04-26T04:32:20.715 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 16 erasure shecprofile 2024-04-26T04:32:20.721 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:20.719+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:20.725 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:20.725 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:20.739 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T04:32:20.822 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:20.819+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:20.909 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:20.910 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:20.922 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:20.920+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:20.924 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T04:32:21.024 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:21.023+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:21.037 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:21.037 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:21.050 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T04:32:21.125 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:21.123+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:21.159 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:21.159 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:21.173 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T04:32:21.227 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:21.224+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:21.284 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:21.284 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:21.298 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T04:32:21.327 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:21.324+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:21.419 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:21.419 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:21.428 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:21.427+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:21.433 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T04:32:21.529 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:21.524+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:21.560 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:21.560 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:21.574 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T04:32:21.630 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:21.627+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:21.699 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:21.699 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:21.713 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T04:32:21.760 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:21.759+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:21.831 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:21.831 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:21.832 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:21.831+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:21.845 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T04:32:21.935 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:21.932+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:22.012 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:22.012 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:22.026 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T04:32:22.035 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:22.032+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:22.137 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:22.135+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:22.143 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:22.143 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:22.157 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T04:32:22.164 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'unique_pool_0' created 2024-04-26T04:32:22.195 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-04-26T04:32:22.237 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:22.232+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:22.282 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:22.282 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:22.297 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T04:32:22.338 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:22.336+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:22.419 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:22.419 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:22.434 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T04:32:22.441 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:22.439+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:22.495 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T04:32:22.542 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:22.540+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:22.554 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:22.554 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:22.568 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T04:32:22.641 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:22.636+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:22.642 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:22.642 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:22.660 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:22.661 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:22.679 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:22.680 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:22.688 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:22.688 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:22.700 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:22.701 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:22.703 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T04:32:22.708 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-26T04:32:22.720 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:22.720 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:22.737 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:22.737 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:22.742 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:22.740+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:22.758 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:22.758 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:22.780 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:22.780 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:22.799 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:22.799 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:22.821 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:22.821 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:22.831 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:22.831 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:32:22.831 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:32:22.831 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [] 2024-04-26T04:32:22.831 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:22.843 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:22.840+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:22.843 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:22.843 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:22.845 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T04:32:22.864 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:22.864 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:22.882 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:22.882 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:22.944 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:22.943+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:22.963 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:22.963 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:22.978 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T04:32:23.005 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:23.005 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:23.034 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:23.034 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:23.045 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:23.043+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:23.066 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:23.067 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:23.145 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:23.143+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:23.151 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:23.151 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:23.151 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:32:23.151 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:32:23.151 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:23.165 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T04:32:23.167 INFO:teuthology.orchestra.run.smithi012.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-04-26T04:32:23.168 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-04-26T04:32:23.246 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:23.244+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:23.298 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:23.298 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:32:23.298 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:32:23.298 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:32:23.299 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.299 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-26T04:32:23.299 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:32:23.299 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 13.823808657000001, 2024-04-26T04:32:23.299 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.261775152, 2024-04-26T04:32:23.299 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.299 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.299 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.299 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.299 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.299 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.299 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 1 2024-04-26T04:32:23.299 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.300 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471107+0000", 2024-04-26T04:32:23.301 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4540000000000001e-06 2024-04-26T04:32:23.301 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.301 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.301 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.301 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471120+0000", 2024-04-26T04:32:23.301 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3217999999999999e-05 2024-04-26T04:32:23.301 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.301 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.301 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.301 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471147+0000", 2024-04-26T04:32:23.302 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6461e-05 2024-04-26T04:32:23.302 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.302 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.302 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.302 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471188+0000", 2024-04-26T04:32:23.302 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1539000000000003e-05 2024-04-26T04:32:23.302 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.302 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.472842+0000", 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016540700000000001 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for peered", 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.472873+0000", 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0300000000000001e-05 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.729650+0000", 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2567770170000001 2024-04-26T04:32:23.303 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.729957+0000", 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00030700500000000001 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.730324+0000", 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00036723099999999998 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732599+0000", 2024-04-26T04:32:23.304 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0022751260000000001 2024-04-26T04:32:23.305 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.305 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.305 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.305 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732706+0000", 2024-04-26T04:32:23.305 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010741 2024-04-26T04:32:23.305 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.305 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.305 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.305 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732804+0000", 2024-04-26T04:32:23.305 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7348000000000004e-05 2024-04-26T04:32:23.305 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.305 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.306 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.306 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732880+0000", 2024-04-26T04:32:23.306 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5972999999999999e-05 2024-04-26T04:32:23.306 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.306 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.306 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.306 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.306 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.306 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:105 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 512~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.306 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.607436+0000", 2024-04-26T04:32:23.306 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.6874776090000001, 2024-04-26T04:32:23.306 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.50341344899999996, 2024-04-26T04:32:23.306 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.307 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.307 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.307 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.307 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.307 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.307 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 105 2024-04-26T04:32:23.307 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.307 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.307 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.307 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.307 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607436+0000", 2024-04-26T04:32:23.307 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.307 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607436+0000", 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607436+0000", 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.1999999999999998e-07 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607444+0000", 2024-04-26T04:32:23.308 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0489999999999998e-06 2024-04-26T04:32:23.309 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.309 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.309 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.309 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607455+0000", 2024-04-26T04:32:23.309 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1855000000000001e-05 2024-04-26T04:32:23.309 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.309 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.309 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.309 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607464+0000", 2024-04-26T04:32:23.309 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2900000000000002e-06 2024-04-26T04:32:23.309 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.309 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.309 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.608572+0000", 2024-04-26T04:32:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0011083250000000001 2024-04-26T04:32:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.608812+0000", 2024-04-26T04:32:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00023972899999999999 2024-04-26T04:32:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.609167+0000", 2024-04-26T04:32:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00035556299999999998 2024-04-26T04:32:23.310 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.311 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.311 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.311 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.612729+0000", 2024-04-26T04:32:23.311 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0035620059999999999 2024-04-26T04:32:23.311 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.311 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.311 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.311 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.110548+0000", 2024-04-26T04:32:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.497818704 2024-04-26T04:32:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.110715+0000", 2024-04-26T04:32:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000166854 2024-04-26T04:32:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.110849+0000", 2024-04-26T04:32:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013415399999999999 2024-04-26T04:32:23.312 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.607530+0000", 2024-04-26T04:32:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.687383327, 2024-04-26T04:32:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.503621765, 2024-04-26T04:32:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.313 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 106 2024-04-26T04:32:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607530+0000", 2024-04-26T04:32:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.314 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607530+0000", 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607531+0000", 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.094e-06 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607767+0000", 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000236178 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.315 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.316 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607788+0000", 2024-04-26T04:32:23.316 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0854999999999999e-05 2024-04-26T04:32:23.316 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.316 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.316 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.316 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607798+0000", 2024-04-26T04:32:23.316 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0529e-05 2024-04-26T04:32:23.316 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.316 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.316 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.316 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.610063+0000", 2024-04-26T04:32:23.317 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0022648149999999999 2024-04-26T04:32:23.317 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.317 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.317 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.317 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.610222+0000", 2024-04-26T04:32:23.317 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000158751 2024-04-26T04:32:23.317 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.317 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.317 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.317 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.610430+0000", 2024-04-26T04:32:23.317 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020761099999999999 2024-04-26T04:32:23.317 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.317 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.318 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.318 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.612742+0000", 2024-04-26T04:32:23.318 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0023125110000000002 2024-04-26T04:32:23.318 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.318 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.318 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.318 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.110993+0000", 2024-04-26T04:32:23.318 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.49825100500000002 2024-04-26T04:32:23.319 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.319 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.319 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.319 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.111083+0000", 2024-04-26T04:32:23.319 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.0206999999999999e-05 2024-04-26T04:32:23.319 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.320 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.320 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.320 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.111152+0000", 2024-04-26T04:32:23.320 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.8208999999999997e-05 2024-04-26T04:32:23.320 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.320 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.320 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.320 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.321 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.321 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.321 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.607819+0000", 2024-04-26T04:32:23.321 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.6870943650000001, 2024-04-26T04:32:23.321 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.50427986199999997, 2024-04-26T04:32:23.321 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.321 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.321 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.322 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.322 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.322 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.322 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 107 2024-04-26T04:32:23.322 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.322 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.322 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.322 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.322 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607819+0000", 2024-04-26T04:32:23.322 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607819+0000", 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607820+0000", 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1299999999999999e-07 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.323 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607826+0000", 2024-04-26T04:32:23.324 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.5089999999999999e-06 2024-04-26T04:32:23.324 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.324 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.324 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.324 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607837+0000", 2024-04-26T04:32:23.324 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0468999999999999e-05 2024-04-26T04:32:23.324 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.324 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.324 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.325 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.607844+0000", 2024-04-26T04:32:23.325 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.6170000000000001e-06 2024-04-26T04:32:23.325 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.325 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.326 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.326 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.611237+0000", 2024-04-26T04:32:23.326 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0033930480000000001 2024-04-26T04:32:23.326 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.326 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.326 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.326 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.611461+0000", 2024-04-26T04:32:23.326 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000223773 2024-04-26T04:32:23.326 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.611829+0000", 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00036794900000000001 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.613296+0000", 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001466698 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.111948+0000", 2024-04-26T04:32:23.327 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.49865248699999998 2024-04-26T04:32:23.328 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.328 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.328 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.328 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.112033+0000", 2024-04-26T04:32:23.328 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4939999999999997e-05 2024-04-26T04:32:23.328 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.328 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.328 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.328 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.112099+0000", 2024-04-26T04:32:23.328 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.5559e-05 2024-04-26T04:32:23.328 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.328 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.328 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.329 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.329 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.329 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.329 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.608002+0000", 2024-04-26T04:32:23.329 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.6869114789999999, 2024-04-26T04:32:23.329 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.50429086499999998, 2024-04-26T04:32:23.329 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.331 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.331 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.331 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.331 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.331 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.331 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 108 2024-04-26T04:32:23.331 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.331 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.331 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.331 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.608002+0000", 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.608002+0000", 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.608003+0000", 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.206e-06 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.332 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.333 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.333 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.608010+0000", 2024-04-26T04:32:23.333 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3560000000000004e-06 2024-04-26T04:32:23.333 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.333 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.333 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.333 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.608022+0000", 2024-04-26T04:32:23.333 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2189e-05 2024-04-26T04:32:23.333 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.333 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.333 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.333 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708163+0000", 2024-04-26T04:32:23.333 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100140939 2024-04-26T04:32:23.334 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.334 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.334 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.334 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708322+0000", 2024-04-26T04:32:23.334 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015874500000000001 2024-04-26T04:32:23.334 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.334 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.334 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.334 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708556+0000", 2024-04-26T04:32:23.334 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00023415900000000001 2024-04-26T04:32:23.335 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.335 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.335 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.335 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708918+0000", 2024-04-26T04:32:23.335 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00036185900000000002 2024-04-26T04:32:23.335 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.335 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.335 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.335 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.815237+0000", 2024-04-26T04:32:23.335 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.106319175 2024-04-26T04:32:23.335 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.336 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.336 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.336 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.112171+0000", 2024-04-26T04:32:23.336 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.29693347599999997 2024-04-26T04:32:23.336 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.336 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.336 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.336 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.112242+0000", 2024-04-26T04:32:23.336 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.1005000000000002e-05 2024-04-26T04:32:23.336 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.112293+0000", 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.0756e-05 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708685+0000", 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.5862282719999996, 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.40735069000000002, 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.337 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.338 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.338 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.338 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.338 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.338 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 114 2024-04-26T04:32:23.338 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.338 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.338 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.338 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.338 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708685+0000", 2024-04-26T04:32:23.339 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.339 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.339 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.339 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.339 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708685+0000", 2024-04-26T04:32:23.339 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.339 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.339 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.339 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.339 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708686+0000", 2024-04-26T04:32:23.340 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.7000000000000004e-07 2024-04-26T04:32:23.340 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.340 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.340 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.340 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708691+0000", 2024-04-26T04:32:23.340 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.7559999999999996e-06 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708701+0000", 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.4900000000000006e-06 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708709+0000", 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2120000000000003e-06 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.341 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.816377+0000", 2024-04-26T04:32:23.342 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.107668476 2024-04-26T04:32:23.342 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.342 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.342 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.342 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.816585+0000", 2024-04-26T04:32:23.342 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020751400000000001 2024-04-26T04:32:23.342 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.342 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.342 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.343 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.816934+0000", 2024-04-26T04:32:23.343 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00034945000000000001 2024-04-26T04:32:23.343 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.343 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.343 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.343 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820872+0000", 2024-04-26T04:32:23.343 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0039378399999999997 2024-04-26T04:32:23.343 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.343 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.343 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.343 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.115912+0000", 2024-04-26T04:32:23.343 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.29503951099999998 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.115983+0000", 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.1625000000000006e-05 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.116036+0000", 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.2145999999999998e-05 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.344 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708719+0000", 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.5861936749999996, 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60283476700000005, 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 115 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.345 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.346 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.346 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708719+0000", 2024-04-26T04:32:23.346 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.346 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.346 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.346 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.346 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708719+0000", 2024-04-26T04:32:23.346 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.346 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.347 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.347 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.347 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708720+0000", 2024-04-26T04:32:23.347 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6700000000000003e-07 2024-04-26T04:32:23.347 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.347 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.347 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.347 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708854+0000", 2024-04-26T04:32:23.347 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013397699999999999 2024-04-26T04:32:23.347 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.347 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.347 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.347 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708866+0000", 2024-04-26T04:32:23.348 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1875000000000001e-05 2024-04-26T04:32:23.348 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.348 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.348 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.348 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708874+0000", 2024-04-26T04:32:23.348 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0600000000000008e-06 2024-04-26T04:32:23.348 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.348 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.348 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.348 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.818243+0000", 2024-04-26T04:32:23.348 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10936897900000001 2024-04-26T04:32:23.349 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.349 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.349 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.349 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.818373+0000", 2024-04-26T04:32:23.349 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012997700000000001 2024-04-26T04:32:23.349 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.349 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.349 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.349 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.818585+0000", 2024-04-26T04:32:23.350 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021234699999999999 2024-04-26T04:32:23.350 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.350 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.350 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.350 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820901+0000", 2024-04-26T04:32:23.350 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0023151669999999999 2024-04-26T04:32:23.350 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.350 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.351 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.351 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311251+0000", 2024-04-26T04:32:23.351 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.49035071699999999 2024-04-26T04:32:23.351 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.351 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.351 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.351 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311421+0000", 2024-04-26T04:32:23.351 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016933399999999999 2024-04-26T04:32:23.352 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.352 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.352 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.352 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311554+0000", 2024-04-26T04:32:23.352 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013366699999999999 2024-04-26T04:32:23.352 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.352 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.352 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.353 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.353 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.353 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.353 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708888+0000", 2024-04-26T04:32:23.353 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.5860249309999999, 2024-04-26T04:32:23.353 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60310785300000003, 2024-04-26T04:32:23.353 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.353 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.354 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.354 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.354 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.354 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.354 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 116 2024-04-26T04:32:23.354 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.354 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.354 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.355 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.355 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708888+0000", 2024-04-26T04:32:23.355 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.355 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.355 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.355 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.355 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708888+0000", 2024-04-26T04:32:23.355 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.355 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.356 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.356 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.356 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708889+0000", 2024-04-26T04:32:23.356 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8700000000000005e-07 2024-04-26T04:32:23.356 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.356 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.356 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.356 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708894+0000", 2024-04-26T04:32:23.357 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.446e-06 2024-04-26T04:32:23.357 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.357 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.357 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.357 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708904+0000", 2024-04-26T04:32:23.357 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7059999999999999e-06 2024-04-26T04:32:23.357 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.358 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.358 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.358 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708912+0000", 2024-04-26T04:32:23.358 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3730000000000004e-06 2024-04-26T04:32:23.358 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.358 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.359 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.359 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.819534+0000", 2024-04-26T04:32:23.359 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.110621994 2024-04-26T04:32:23.359 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.359 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.359 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.359 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.819754+0000", 2024-04-26T04:32:23.359 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00022034999999999999 2024-04-26T04:32:23.360 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.360 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.360 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.360 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820112+0000", 2024-04-26T04:32:23.360 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00035776299999999998 2024-04-26T04:32:23.360 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.360 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.360 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.361 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821816+0000", 2024-04-26T04:32:23.361 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001704346 2024-04-26T04:32:23.361 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.361 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.361 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.362 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311719+0000", 2024-04-26T04:32:23.362 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.48990303600000001 2024-04-26T04:32:23.362 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.362 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.362 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.362 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311876+0000", 2024-04-26T04:32:23.362 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015653800000000001 2024-04-26T04:32:23.362 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.363 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.363 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.363 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311996+0000", 2024-04-26T04:32:23.363 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012051400000000001 2024-04-26T04:32:23.363 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.363 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.363 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.363 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.365 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.365 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.365 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:32:23.365 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.5859907260000004, 2024-04-26T04:32:23.366 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60751630199999995, 2024-04-26T04:32:23.366 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.366 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.366 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.366 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.366 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.366 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.366 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 117 2024-04-26T04:32:23.366 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.366 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.366 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.367 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.367 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:32:23.367 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.367 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.367 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.367 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.367 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:32:23.367 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.367 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.367 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.368 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.368 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708923+0000", 2024-04-26T04:32:23.368 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3300000000000002e-07 2024-04-26T04:32:23.368 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.369 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.369 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.369 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708929+0000", 2024-04-26T04:32:23.369 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.9959999999999999e-06 2024-04-26T04:32:23.369 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.369 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.369 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.369 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708941+0000", 2024-04-26T04:32:23.369 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2085e-05 2024-04-26T04:32:23.369 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.369 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.370 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.370 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708948+0000", 2024-04-26T04:32:23.370 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.8170000000000001e-06 2024-04-26T04:32:23.370 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.370 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.370 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.370 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820967+0000", 2024-04-26T04:32:23.370 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.112019008 2024-04-26T04:32:23.370 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.371 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.371 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.371 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821107+0000", 2024-04-26T04:32:23.371 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000139825 2024-04-26T04:32:23.371 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.371 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.371 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.371 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821309+0000", 2024-04-26T04:32:23.371 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020262899999999999 2024-04-26T04:32:23.371 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.371 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.371 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.371 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.822530+0000", 2024-04-26T04:32:23.372 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0012206129999999999 2024-04-26T04:32:23.372 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.372 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.372 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.372 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316288+0000", 2024-04-26T04:32:23.372 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.49375772400000001 2024-04-26T04:32:23.372 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.372 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.372 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.372 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316375+0000", 2024-04-26T04:32:23.372 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7312999999999996e-05 2024-04-26T04:32:23.372 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.372 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.373 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.373 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316439+0000", 2024-04-26T04:32:23.373 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3658999999999995e-05 2024-04-26T04:32:23.373 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.373 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.373 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.373 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.373 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.373 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:155 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.373 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.738885+0000", 2024-04-26T04:32:23.373 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.5560280119999996, 2024-04-26T04:32:23.373 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.41317219999999999, 2024-04-26T04:32:23.373 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.374 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.374 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.374 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.374 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.374 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.374 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 155 2024-04-26T04:32:23.374 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.374 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.374 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.374 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.374 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.738885+0000", 2024-04-26T04:32:23.374 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.374 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.738885+0000", 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.738886+0000", 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.9699999999999994e-07 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.738977+0000", 2024-04-26T04:32:23.375 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.1260000000000004e-05 2024-04-26T04:32:23.376 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.376 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.376 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.376 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.738992+0000", 2024-04-26T04:32:23.376 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5026e-05 2024-04-26T04:32:23.376 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.376 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.376 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.376 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739002+0000", 2024-04-26T04:32:23.376 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.9049999999999992e-06 2024-04-26T04:32:23.376 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.376 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.376 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.377 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.740749+0000", 2024-04-26T04:32:23.377 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0017464939999999999 2024-04-26T04:32:23.377 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.377 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.377 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.377 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.740884+0000", 2024-04-26T04:32:23.377 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000135437 2024-04-26T04:32:23.378 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.378 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.378 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.378 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.741091+0000", 2024-04-26T04:32:23.378 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020633199999999999 2024-04-26T04:32:23.378 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.378 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.378 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.744778+0000", 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.003687241 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.151909+0000", 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.40713135900000003 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.151993+0000", 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4196999999999994e-05 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.379 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.152057+0000", 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3952e-05 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:156 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739020+0000", 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.5558932409999997, 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.41323475799999998, 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.380 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 156 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739020+0000", 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.381 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739020+0000", 2024-04-26T04:32:23.382 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.382 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.382 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.382 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.382 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739021+0000", 2024-04-26T04:32:23.382 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.8599999999999996e-07 2024-04-26T04:32:23.382 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.382 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.382 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.382 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739027+0000", 2024-04-26T04:32:23.382 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.879e-06 2024-04-26T04:32:23.382 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.382 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.383 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.383 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739037+0000", 2024-04-26T04:32:23.383 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0443e-05 2024-04-26T04:32:23.383 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.383 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.383 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.383 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739045+0000", 2024-04-26T04:32:23.383 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5460000000000003e-06 2024-04-26T04:32:23.383 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.383 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.383 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.383 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.741930+0000", 2024-04-26T04:32:23.383 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0028855819999999998 2024-04-26T04:32:23.384 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.384 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.384 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.384 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.742072+0000", 2024-04-26T04:32:23.384 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000141753 2024-04-26T04:32:23.384 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.384 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.384 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.384 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.742268+0000", 2024-04-26T04:32:23.384 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000196326 2024-04-26T04:32:23.384 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.384 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.384 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.744784+0000", 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0025155120000000001 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.152133+0000", 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.40734946700000002 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.152204+0000", 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0544999999999996e-05 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.385 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.386 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.386 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.152255+0000", 2024-04-26T04:32:23.386 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.0719e-05 2024-04-26T04:32:23.386 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.386 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.386 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.386 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.386 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.386 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.386 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739057+0000", 2024-04-26T04:32:23.386 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.5558560779999997, 2024-04-26T04:32:23.386 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51358532199999996, 2024-04-26T04:32:23.386 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.387 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.387 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.387 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.387 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.387 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.387 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 157 2024-04-26T04:32:23.387 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.387 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.387 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.387 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.387 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739057+0000", 2024-04-26T04:32:23.387 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.387 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.388 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.388 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.388 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739057+0000", 2024-04-26T04:32:23.388 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.388 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.388 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.388 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.388 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739058+0000", 2024-04-26T04:32:23.388 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3e-07 2024-04-26T04:32:23.388 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.389 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.389 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.389 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739063+0000", 2024-04-26T04:32:23.389 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.22e-06 2024-04-26T04:32:23.389 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.389 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.389 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.389 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739073+0000", 2024-04-26T04:32:23.389 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.9229999999999997e-06 2024-04-26T04:32:23.390 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.390 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.390 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.390 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739080+0000", 2024-04-26T04:32:23.390 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.1659999999999997e-06 2024-04-26T04:32:23.390 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.390 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.390 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.390 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.742847+0000", 2024-04-26T04:32:23.390 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0037669980000000001 2024-04-26T04:32:23.390 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.390 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.743008+0000", 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016099199999999999 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.743230+0000", 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00022182699999999999 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.745704+0000", 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0024736519999999998 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.391 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252490+0000", 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.50678671500000005 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252576+0000", 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.5630000000000005e-05 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252642+0000", 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6469000000000003e-05 2024-04-26T04:32:23.392 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739091+0000", 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.555821656, 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51402278300000004, 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.393 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 158 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739091+0000", 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739091+0000", 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.394 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739092+0000", 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.99e-07 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739179+0000", 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6510999999999999e-05 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739190+0000", 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1629e-05 2024-04-26T04:32:23.395 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739201+0000", 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0739e-05 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.743817+0000", 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0046161170000000003 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.743935+0000", 2024-04-26T04:32:23.396 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000117661 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.744145+0000", 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021043900000000001 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.746703+0000", 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0025576919999999999 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.397 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252837+0000", 2024-04-26T04:32:23.398 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.506133783 2024-04-26T04:32:23.398 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.398 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.398 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.398 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252994+0000", 2024-04-26T04:32:23.398 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015751200000000001 2024-04-26T04:32:23.398 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.398 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.398 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.398 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253114+0000", 2024-04-26T04:32:23.398 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012000100000000001 2024-04-26T04:32:23.398 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.398 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.399 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.399 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.399 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.399 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.399 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739215+0000", 2024-04-26T04:32:23.399 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.5556984849999997, 2024-04-26T04:32:23.399 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51429833800000002, 2024-04-26T04:32:23.399 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.399 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.400 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.400 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.400 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.400 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.400 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 159 2024-04-26T04:32:23.400 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.400 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.400 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.400 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.400 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739215+0000", 2024-04-26T04:32:23.400 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.400 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.400 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739215+0000", 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739215+0000", 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8599999999999997e-07 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739222+0000", 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.1929999999999998e-06 2024-04-26T04:32:23.401 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.402 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.402 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.402 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739235+0000", 2024-04-26T04:32:23.402 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3815000000000001e-05 2024-04-26T04:32:23.402 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.402 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.402 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.402 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739245+0000", 2024-04-26T04:32:23.402 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.0929999999999992e-06 2024-04-26T04:32:23.402 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.744841+0000", 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0055962429999999999 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.744959+0000", 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00011796200000000001 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.745168+0000", 2024-04-26T04:32:23.403 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020875099999999999 2024-04-26T04:32:23.404 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.404 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.404 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.404 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.747905+0000", 2024-04-26T04:32:23.404 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0027373330000000002 2024-04-26T04:32:23.404 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.404 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.404 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.404 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253250+0000", 2024-04-26T04:32:23.404 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.50534521399999999 2024-04-26T04:32:23.404 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.404 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.404 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253403+0000", 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015304899999999999 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253513+0000", 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000109899 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:160 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739257+0000", 2024-04-26T04:32:23.405 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.5556564980000003, 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51462529499999998, 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 160 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739257+0000", 2024-04-26T04:32:23.406 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.407 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.407 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.407 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.407 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739257+0000", 2024-04-26T04:32:23.407 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.407 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.407 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.407 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.407 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739257+0000", 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3600000000000003e-07 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739264+0000", 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.2360000000000001e-06 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739274+0000", 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.9960000000000005e-06 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.408 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739281+0000", 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.1300000000000003e-06 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.745763+0000", 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0064821219999999999 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.745879+0000", 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000116348 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.409 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.410 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.410 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.746176+0000", 2024-04-26T04:32:23.410 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00029663299999999998 2024-04-26T04:32:23.410 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.410 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.410 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.410 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.046853+0000", 2024-04-26T04:32:23.410 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30067709799999998 2024-04-26T04:32:23.410 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.410 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.410 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.410 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253632+0000", 2024-04-26T04:32:23.410 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20677860000000001 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253776+0000", 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014417 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253882+0000", 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000106226 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.411 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:161 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.555621371, 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51490446599999995, 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 161 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.412 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6599999999999996e-07 2024-04-26T04:32:23.413 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739378+0000", 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.5968000000000001e-05 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739390+0000", 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1595e-05 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739398+0000", 2024-04-26T04:32:23.414 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0029999999999994e-06 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.746760+0000", 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0073621420000000003 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.746876+0000", 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000115789 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.747088+0000", 2024-04-26T04:32:23.415 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021234800000000001 2024-04-26T04:32:23.416 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.416 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.416 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.416 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252667+0000", 2024-04-26T04:32:23.416 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.50557901500000002 2024-04-26T04:32:23.416 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.416 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.416 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.416 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254031+0000", 2024-04-26T04:32:23.416 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001363518 2024-04-26T04:32:23.416 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.416 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.416 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254124+0000", 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3376999999999997e-05 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254196+0000", 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2045000000000006e-05 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.417 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739429+0000", 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.5554841880000003, 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51514356699999997, 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 162 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.418 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739429+0000", 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739429+0000", 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739430+0000", 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8999999999999995e-07 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.419 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739435+0000", 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.5969999999999999e-06 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739446+0000", 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0535e-05 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739453+0000", 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.4800000000000004e-06 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.420 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.943028+0000", 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20357512 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.943276+0000", 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024770199999999998 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.943640+0000", 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00036409399999999997 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.421 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252675+0000", 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30903449199999999 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254318+0000", 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016429179999999999 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254463+0000", 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000145623 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.422 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254573+0000", 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010921600000000001 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.5554475270000001, 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.515346639, 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.423 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 163 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.424 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.1699999999999997e-07 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739471+0000", 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.9219999999999996e-06 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.425 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739487+0000", 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5435999999999999e-05 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739494+0000", 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2899999999999997e-06 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.944605+0000", 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20511091200000001 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.426 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.944825+0000", 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00022035499999999999 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.945203+0000", 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00037760499999999999 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252681+0000", 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.307478311 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.427 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254680+0000", 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0019988129999999999 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254756+0000", 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5914999999999994e-05 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254812+0000", 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.6363000000000002e-05 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.428 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.5554073900000001, 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.61583993999999997, 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 164 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.429 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0399999999999995e-07 2024-04-26T04:32:23.430 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739594+0000", 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7046000000000007e-05 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739605+0000", 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1576000000000001e-05 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739613+0000", 2024-04-26T04:32:23.431 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1459999999999996e-06 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.046981+0000", 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30736738000000002 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.047229+0000", 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024846999999999998 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.047619+0000", 2024-04-26T04:32:23.432 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00038963799999999999 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254219+0000", 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20659982700000001 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355048+0000", 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100829484 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355225+0000", 2024-04-26T04:32:23.433 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017717399999999999 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355346+0000", 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000120495 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.555285875, 2024-04-26T04:32:23.434 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.71903434899999996, 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 165 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.435 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739628+0000", 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7000000000000004e-07 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739645+0000", 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6901999999999998e-05 2024-04-26T04:32:23.436 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739656+0000", 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0556e-05 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739667+0000", 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1535000000000001e-05 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355474+0000", 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.61580721299999996 2024-04-26T04:32:23.437 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355612+0000", 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000137687 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355816+0000", 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020451900000000001 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.357276+0000", 2024-04-26T04:32:23.438 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0014599179999999999 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458392+0000", 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101115921 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458547+0000", 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015461199999999999 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458662+0000", 2024-04-26T04:32:23.439 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000114716 2024-04-26T04:32:23.440 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.440 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.440 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.440 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:23.440 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:23.440 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:23.441 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T04:32:23.442 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:23.436+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:23.543 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:23.543+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:23.573 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:23.573 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:23.573 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:32:23.573 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:23.589 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T04:32:23.644 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:23.640+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:23.711 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:32:23.711 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:32:23.711 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:23.725 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T04:32:23.770 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:23.768+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:23.845 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:32:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:32:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:32:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:42 1.0 e26/23 1:30306672:devicehealth::main.db.0000000000000000:head v 26'28, mlcod=26'26)", 2024-04-26T04:32:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:32:23.845 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.622599459, 2024-04-26T04:32:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201587136, 2024-04-26T04:32:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:32:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.846 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220676+0000", 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.843e-06 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220687+0000", 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1727e-05 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.847 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220715+0000", 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8033000000000001e-05 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.320857+0000", 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014171400000001 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.421143+0000", 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100285902 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.848 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.421175+0000", 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2413999999999998e-05 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.422168+0000", 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0009931269999999999 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.422261+0000", 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2375999999999998e-05 2024-04-26T04:32:23.849 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:55 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'33, mlcod=27'29)", 2024-04-26T04:32:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:32:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.299209596000001, 2024-04-26T04:32:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.169473288, 2024-04-26T04:32:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.850 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544065+0000", 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.77e-06 2024-04-26T04:32:23.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544195+0000", 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012971599999999999 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544219+0000", 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3476e-05 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.644316+0000", 2024-04-26T04:32:23.852 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100097189 2024-04-26T04:32:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.711737+0000", 2024-04-26T04:32:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.067420846000000006 2024-04-26T04:32:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.711770+0000", 2024-04-26T04:32:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2959000000000001e-05 2024-04-26T04:32:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.713428+0000", 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016584130000000001 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.713537+0000", 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010891900000000001 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:108 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'70, mlcod=27'64)", 2024-04-26T04:32:23.854 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 7.0348154330000003, 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30260933299999998, 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.855 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.37e-07 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808464+0000", 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8599999999999998e-06 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.856 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808474+0000", 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.9830000000000008e-06 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908648+0000", 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10017390299999999 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908808+0000", 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000159576 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.857 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908845+0000", 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7026000000000001e-05 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110983+0000", 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20213835199999999 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111067+0000", 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3996e-05 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.858 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:109 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'71, mlcod=27'64)", 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.9345807099999996, 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202472819, 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:32:23.859 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908694+0000", 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.144e-06 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.860 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908751+0000", 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7268999999999997e-05 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908769+0000", 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8363e-05 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908780+0000", 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.096e-05 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.861 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.909341+0000", 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00056057500000000001 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.909371+0000", 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0357000000000001e-05 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111074+0000", 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20170292500000001 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.862 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111165+0000", 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.1225999999999997e-05 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:110 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'72, mlcod=27'64)", 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.9344458180000004, 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20385624999999999, 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.863 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908828+0000", 2024-04-26T04:32:23.864 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9400000000000005e-07 2024-04-26T04:32:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908836+0000", 2024-04-26T04:32:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1929999999999993e-06 2024-04-26T04:32:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908847+0000", 2024-04-26T04:32:23.865 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0703000000000001e-05 2024-04-26T04:32:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908855+0000", 2024-04-26T04:32:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7829999999999999e-06 2024-04-26T04:32:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009054+0000", 2024-04-26T04:32:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10019966900000001 2024-04-26T04:32:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.866 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009089+0000", 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4433e-05 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112582+0000", 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103492973 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112684+0000", 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000101802 2024-04-26T04:32:23.867 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:111 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'73, mlcod=27'64)", 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.9344068879999998, 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.205422306, 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.868 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908867+0000", 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.03e-07 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.869 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908871+0000", 2024-04-26T04:32:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5800000000000002e-06 2024-04-26T04:32:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908883+0000", 2024-04-26T04:32:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1507e-05 2024-04-26T04:32:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009022+0000", 2024-04-26T04:32:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100139593 2024-04-26T04:32:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.870 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.109703+0000", 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100680741 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.109738+0000", 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5296000000000002e-05 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114207+0000", 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0044682239999999998 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.871 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114289+0000", 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2062000000000001e-05 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:112 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'74, mlcod=27'64)", 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.8342127660000003, 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105317487, 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.872 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.873 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009061+0000", 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.057e-06 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009128+0000", 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6859999999999993e-05 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009147+0000", 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8196e-05 2024-04-26T04:32:23.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009157+0000", 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0743e-05 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110254+0000", 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101096941 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110311+0000", 2024-04-26T04:32:23.875 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6718999999999997e-05 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114296+0000", 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0039847709999999998 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114378+0000", 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2200000000000006e-05 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:114 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'76, mlcod=27'64)", 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.8340635169999997, 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105329463, 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.877 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.53e-07 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009215+0000", 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7099999999999998e-06 2024-04-26T04:32:23.878 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009223+0000", 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5599999999999994e-06 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009229+0000", 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0800000000000002e-06 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111865+0000", 2024-04-26T04:32:23.879 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102636071 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111941+0000", 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5615e-05 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114468+0000", 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002526875 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114539+0000", 2024-04-26T04:32:23.880 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1198999999999994e-05 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:115 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'77, mlcod=27'64)", 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.8340356440000001, 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105390922, 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.881 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.34e-07 2024-04-26T04:32:23.882 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009293+0000", 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4954999999999998e-05 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009304+0000", 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1263999999999999e-05 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009311+0000", 2024-04-26T04:32:23.883 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6630000000000004e-06 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112802+0000", 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10349130600000001 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112843+0000", 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0472999999999997e-05 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114547+0000", 2024-04-26T04:32:23.884 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001704552 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114628+0000", 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1374999999999999e-05 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:116 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'78, mlcod=27'64)", 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.8339523910000004, 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105877241, 2024-04-26T04:32:23.885 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.886 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.46e-07 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009326+0000", 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5040000000000004e-06 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009334+0000", 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1929999999999993e-06 2024-04-26T04:32:23.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009340+0000", 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.3729999999999998e-06 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.113595+0000", 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104254889 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.113649+0000", 2024-04-26T04:32:23.888 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3887000000000001e-05 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115105+0000", 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014560969999999999 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115198+0000", 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2851999999999998e-05 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.889 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.890 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.890 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:117 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'79, mlcod=27'64)", 2024-04-26T04:32:23.890 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:32:23.890 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.8339249080000002, 2024-04-26T04:32:23.890 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10639752, 2024-04-26T04:32:23.890 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:32:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:32:23.891 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009349+0000", 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7300000000000002e-07 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009353+0000", 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5970000000000002e-06 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.892 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009361+0000", 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0600000000000008e-06 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009381+0000", 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9533000000000001e-05 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114709+0000", 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105328753 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.893 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114745+0000", 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5976000000000002e-05 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115683+0000", 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00093774899999999998 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115746+0000", 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2578999999999998e-05 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.894 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:155 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'109, mlcod=27'105)", 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.0037371349999997, 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20562082700000001, 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.895 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839537+0000", 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7400000000000003e-07 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.896 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839638+0000", 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000101365 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839653+0000", 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4874e-05 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939797+0000", 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014414100000001 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.897 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.040092+0000", 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100295305 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.040151+0000", 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8295999999999999e-05 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045057+0000", 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0049061390000000003 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.898 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045157+0000", 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000100133 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:156 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'110, mlcod=27'105)", 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.9034416580000002, 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10750937100000001, 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.899 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.900 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939833+0000", 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1030000000000001e-06 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939849+0000", 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6005999999999999e-05 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939866+0000", 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7513e-05 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.901 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939876+0000", 2024-04-26T04:32:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0154e-05 2024-04-26T04:32:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.041330+0000", 2024-04-26T04:32:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101453864 2024-04-26T04:32:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.041398+0000", 2024-04-26T04:32:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7617e-05 2024-04-26T04:32:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.902 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047245+0000", 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058470140000000002 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047341+0000", 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6100000000000005e-05 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.903 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:157 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'111, mlcod=27'105)", 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.9033415189999996, 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107487887, 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.904 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2300000000000001e-07 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939942+0000", 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4369999999999996e-06 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.905 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939952+0000", 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.075e-05 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939963+0000", 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0069e-05 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.042203+0000", 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102240215 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.906 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.042259+0000", 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5955999999999998e-05 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047353+0000", 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0050939080000000003 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047420+0000", 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6928999999999996e-05 2024-04-26T04:32:23.907 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:158 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'112, mlcod=27'105)", 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.9032989330000003, 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.209867842, 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.908 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939975+0000", 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8599999999999999e-07 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940084+0000", 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000109746 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940099+0000", 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4392999999999999e-05 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940110+0000", 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1236999999999999e-05 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.910 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043002+0000", 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102892262 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043054+0000", 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1922999999999997e-05 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149774+0000", 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10672018599999999 2024-04-26T04:32:23.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149842+0000", 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7708999999999998e-05 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:159 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'113, mlcod=27'105)", 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.9031503110000001, 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20976125000000001, 2024-04-26T04:32:23.912 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.913 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7000000000000005e-07 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940131+0000", 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6369999999999999e-06 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940143+0000", 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1603000000000001e-05 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.914 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940165+0000", 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2042999999999999e-05 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043777+0000", 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103612145 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043807+0000", 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0562999999999999e-05 2024-04-26T04:32:23.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149845+0000", 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10603802900000001 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149884+0000", 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8659999999999999e-05 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.916 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:160 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'114, mlcod=27'105)", 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.9030998050000001, 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20974973299999999, 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.917 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940174+0000", 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8999999999999998e-07 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940183+0000", 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6970000000000005e-06 2024-04-26T04:32:23.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940194+0000", 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0983e-05 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940201+0000", 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4400000000000002e-06 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.044326+0000", 2024-04-26T04:32:23.919 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104124984 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.044377+0000", 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1249999999999999e-05 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149886+0000", 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10550875999999999 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149923+0000", 2024-04-26T04:32:23.920 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7329000000000001e-05 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:161 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'115, mlcod=27'105)", 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.9030638619999998, 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.209755314, 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.921 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:32:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:32:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940210+0000", 2024-04-26T04:32:23.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.67e-07 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940330+0000", 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012061900000000001 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940344+0000", 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.367e-05 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940355+0000", 2024-04-26T04:32:23.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.083e-05 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045239+0000", 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104883873 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045278+0000", 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9904000000000003e-05 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.924 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149925+0000", 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104646221 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149965+0000", 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9929999999999999e-05 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:162 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'116, mlcod=27'105)", 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.8996001299999996, 2024-04-26T04:32:23.925 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.206336451, 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943675+0000", 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8640000000000001e-06 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943692+0000", 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7503000000000002e-05 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943720+0000", 2024-04-26T04:32:23.927 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7569999999999999e-05 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943734+0000", 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3596e-05 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.046517+0000", 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102783897 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.928 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.046569+0000", 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1706000000000003e-05 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149968+0000", 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10339902199999999 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150009+0000", 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1292999999999999e-05 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.929 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:163 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'117, mlcod=27'105)", 2024-04-26T04:32:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:32:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.8980990599999998, 2024-04-26T04:32:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20488266699999999, 2024-04-26T04:32:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:23.930 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945176+0000", 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8470000000000001e-06 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:23.931 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945193+0000", 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7013000000000001e-05 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945221+0000", 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7670000000000001e-05 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945238+0000", 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7113e-05 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:23.932 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.147614+0000", 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202375893 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.147669+0000", 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5729000000000001e-05 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150011+0000", 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002342083 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:23.933 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150057+0000", 2024-04-26T04:32:23.934 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5318999999999998e-05 2024-04-26T04:32:23.934 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.934 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.934 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.934 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:23.934 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:23.934 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:23.936 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T04:32:23.936 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:23.930+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:24.037 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:24.032+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:24.104 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:24.104 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:24.104 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:32:24.105 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:24.119 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T04:32:24.137 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:24.135+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:24.203 DEBUG:teuthology.orchestra.run.smithi178:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2024-04-26T04:32:24.226 INFO:tasks.rados.rados.0.smithi178.stdout:adding op weight read -> 100 2024-04-26T04:32:24.226 INFO:tasks.rados.rados.0.smithi178.stdout:adding op weight delete -> 50 2024-04-26T04:32:24.226 INFO:tasks.rados.rados.0.smithi178.stdout:adding op weight snap_create -> 50 2024-04-26T04:32:24.227 INFO:tasks.rados.rados.0.smithi178.stdout:adding op weight snap_remove -> 50 2024-04-26T04:32:24.227 INFO:tasks.rados.rados.0.smithi178.stdout:adding op weight rollback -> 50 2024-04-26T04:32:24.227 INFO:tasks.rados.rados.0.smithi178.stdout:adding op weight setattr -> 25 2024-04-26T04:32:24.227 INFO:tasks.rados.rados.0.smithi178.stdout:adding op weight rmattr -> 25 2024-04-26T04:32:24.227 INFO:tasks.rados.rados.0.smithi178.stdout:adding op weight copy_from -> 50 2024-04-26T04:32:24.227 INFO:tasks.rados.rados.0.smithi178.stdout:adding op weight append -> 100 2024-04-26T04:32:24.228 INFO:tasks.rados.rados.0.smithi178.stdout:ceph version 19.0.0-2457-gddeaa917 (ddeaa91797339a19172c3036ff48cca58c12f448) squid (dev) 2024-04-26T04:32:24.228 INFO:tasks.rados.rados.0.smithi178.stdout:Configuration: 2024-04-26T04:32:24.228 INFO:tasks.rados.rados.0.smithi178.stdout: Number of operations: 400 2024-04-26T04:32:24.228 INFO:tasks.rados.rados.0.smithi178.stdout: Number of objects: 50 2024-04-26T04:32:24.228 INFO:tasks.rados.rados.0.smithi178.stdout: Max in flight operations: 16 2024-04-26T04:32:24.228 INFO:tasks.rados.rados.0.smithi178.stdout: Object size (in bytes): 4000000 2024-04-26T04:32:24.229 INFO:tasks.rados.rados.0.smithi178.stdout: Write stride min: 400000 2024-04-26T04:32:24.229 INFO:tasks.rados.rados.0.smithi178.stdout: Write stride max: 800000 2024-04-26T04:32:24.238 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:24.235+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:24.245 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:24.245 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:24.245 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:32:24.245 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:32:24.246 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:24.259 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T04:32:24.340 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:24.338+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:24.382 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:24.382 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:32:24.383 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:32:24.383 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [] 2024-04-26T04:32:24.383 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:24.397 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T04:32:24.441 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:24.436+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:24.522 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:24.522 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:24.522 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:32:24.522 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:24.535 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T04:32:24.541 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:24.536+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:24.644 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:24.640+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:24.659 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:24.660 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:24.660 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:32:24.660 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:32:24.661 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:24.672 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T04:32:24.745 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:24.740+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:24.797 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:24.797 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:32:24.797 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:32:24.798 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [] 2024-04-26T04:32:24.798 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:24.811 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T04:32:24.854 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:24.852+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:24.946 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:24.944+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:24.976 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:24.977 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:24.977 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:32:24.977 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:24.991 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T04:32:25.047 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:25.046+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:25.091 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T04:32:25.148 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:25.144+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:25.157 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:25.158 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:25.158 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:32:25.158 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:32:25.158 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:25.172 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T04:32:25.197 INFO:tasks.rados.rados.0.smithi178.stdout:1: write initial oid 1 2024-04-26T04:32:25.197 INFO:tasks.rados.rados.0.smithi178.stdout:1: seq_num 0 ranges {0=32768} 2024-04-26T04:32:25.197 INFO:tasks.rados.rados.0.smithi178.stdout:1: writing smithi17829308-1 from 0 to 32768 tid 1 2024-04-26T04:32:25.197 INFO:tasks.rados.rados.0.smithi178.stdout:2: write initial oid 2 2024-04-26T04:32:25.197 INFO:tasks.rados.rados.0.smithi178.stdout:2: seq_num 1 ranges {0=32768} 2024-04-26T04:32:25.197 INFO:tasks.rados.rados.0.smithi178.stdout:2: writing smithi17829308-2 from 0 to 32768 tid 1 2024-04-26T04:32:25.197 INFO:tasks.rados.rados.0.smithi178.stdout:3: write initial oid 3 2024-04-26T04:32:25.198 INFO:tasks.rados.rados.0.smithi178.stdout:3: seq_num 2 ranges {0=49152} 2024-04-26T04:32:25.198 INFO:tasks.rados.rados.0.smithi178.stdout:3: writing smithi17829308-3 from 0 to 49152 tid 1 2024-04-26T04:32:25.198 INFO:tasks.rados.rados.0.smithi178.stdout:4: write initial oid 4 2024-04-26T04:32:25.198 INFO:tasks.rados.rados.0.smithi178.stdout:4: seq_num 3 ranges {0=65536} 2024-04-26T04:32:25.198 INFO:tasks.rados.rados.0.smithi178.stdout:4: writing smithi17829308-4 from 0 to 65536 tid 1 2024-04-26T04:32:25.199 INFO:tasks.rados.rados.0.smithi178.stdout:5: write initial oid 5 2024-04-26T04:32:25.199 INFO:tasks.rados.rados.0.smithi178.stdout:5: seq_num 4 ranges {0=81920} 2024-04-26T04:32:25.199 INFO:tasks.rados.rados.0.smithi178.stdout:5: writing smithi17829308-5 from 0 to 81920 tid 1 2024-04-26T04:32:25.199 INFO:tasks.rados.rados.0.smithi178.stdout:6: write initial oid 6 2024-04-26T04:32:25.199 INFO:tasks.rados.rados.0.smithi178.stdout:6: seq_num 5 ranges {0=98304} 2024-04-26T04:32:25.200 INFO:tasks.rados.rados.0.smithi178.stdout:6: writing smithi17829308-6 from 0 to 98304 tid 1 2024-04-26T04:32:25.200 INFO:tasks.rados.rados.0.smithi178.stdout:7: write initial oid 7 2024-04-26T04:32:25.200 INFO:tasks.rados.rados.0.smithi178.stdout:7: seq_num 6 ranges {0=114688} 2024-04-26T04:32:25.201 INFO:tasks.rados.rados.0.smithi178.stdout:7: writing smithi17829308-7 from 0 to 114688 tid 1 2024-04-26T04:32:25.202 INFO:tasks.rados.rados.0.smithi178.stdout:8: write initial oid 8 2024-04-26T04:32:25.202 INFO:tasks.rados.rados.0.smithi178.stdout:8: seq_num 7 ranges {0=131072} 2024-04-26T04:32:25.202 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-26T04:32:25.203 INFO:tasks.rados.rados.0.smithi178.stdout:8: writing smithi17829308-8 from 0 to 131072 tid 1 2024-04-26T04:32:25.203 INFO:tasks.rados.rados.0.smithi178.stdout:9: write initial oid 9 2024-04-26T04:32:25.203 INFO:tasks.rados.rados.0.smithi178.stdout:9: seq_num 8 ranges {0=147456} 2024-04-26T04:32:25.204 INFO:tasks.rados.rados.0.smithi178.stdout:9: writing smithi17829308-9 from 0 to 147456 tid 1 2024-04-26T04:32:25.204 INFO:tasks.rados.rados.0.smithi178.stdout:10: write initial oid 10 2024-04-26T04:32:25.204 INFO:tasks.rados.rados.0.smithi178.stdout:10: seq_num 9 ranges {0=163840} 2024-04-26T04:32:25.206 INFO:tasks.rados.rados.0.smithi178.stdout:10: writing smithi17829308-10 from 0 to 163840 tid 1 2024-04-26T04:32:25.206 INFO:tasks.rados.rados.0.smithi178.stdout:11: write initial oid 11 2024-04-26T04:32:25.206 INFO:tasks.rados.rados.0.smithi178.stdout:11: seq_num 10 ranges {0=180224} 2024-04-26T04:32:25.209 INFO:tasks.rados.rados.0.smithi178.stdout:11: writing smithi17829308-11 from 0 to 180224 tid 1 2024-04-26T04:32:25.209 INFO:tasks.rados.rados.0.smithi178.stdout:12: write initial oid 12 2024-04-26T04:32:25.210 INFO:tasks.rados.rados.0.smithi178.stdout:12: seq_num 11 ranges {0=196608} 2024-04-26T04:32:25.210 INFO:tasks.rados.rados.0.smithi178.stdout:12: writing smithi17829308-12 from 0 to 196608 tid 1 2024-04-26T04:32:25.210 INFO:tasks.rados.rados.0.smithi178.stdout:13: write initial oid 13 2024-04-26T04:32:25.210 INFO:tasks.rados.rados.0.smithi178.stdout:13: seq_num 12 ranges {0=212992} 2024-04-26T04:32:25.211 INFO:tasks.rados.rados.0.smithi178.stdout:13: writing smithi17829308-13 from 0 to 212992 tid 1 2024-04-26T04:32:25.211 INFO:tasks.rados.rados.0.smithi178.stdout:14: write initial oid 14 2024-04-26T04:32:25.211 INFO:tasks.rados.rados.0.smithi178.stdout:14: seq_num 13 ranges {0=229376} 2024-04-26T04:32:25.212 INFO:tasks.rados.rados.0.smithi178.stdout:14: writing smithi17829308-14 from 0 to 229376 tid 1 2024-04-26T04:32:25.213 INFO:tasks.rados.rados.0.smithi178.stdout:15: write initial oid 15 2024-04-26T04:32:25.213 INFO:tasks.rados.rados.0.smithi178.stdout:15: seq_num 14 ranges {0=245760} 2024-04-26T04:32:25.215 INFO:tasks.rados.rados.0.smithi178.stdout:15: writing smithi17829308-15 from 0 to 245760 tid 1 2024-04-26T04:32:25.215 INFO:tasks.rados.rados.0.smithi178.stdout:16: write initial oid 16 2024-04-26T04:32:25.215 INFO:tasks.rados.rados.0.smithi178.stdout:16: seq_num 15 ranges {0=262144} 2024-04-26T04:32:25.217 INFO:tasks.rados.rados.0.smithi178.stdout:16: writing smithi17829308-16 from 0 to 262144 tid 1 2024-04-26T04:32:25.217 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:25.248 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:25.247+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:25.249 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:25.250 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:25.270 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:25.271 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:25.289 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:25.289 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:25.310 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:25.310 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:25.321 INFO:tasks.rados.rados.0.smithi178.stdout:14: finishing write tid 1 to smithi17829308-14 2024-04-26T04:32:25.344 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:25.344 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:32:25.344 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:32:25.344 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:32:25.344 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.4s5 32/30 ECSubWrite(tid=0, reqid=client.4670.0:10, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.204931+0000", 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.14034144200000001, 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001160173, 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204931+0000", 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.345 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.346 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:25.346 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204931+0000", 2024-04-26T04:32:25.346 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.346 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.346 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.346 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:25.346 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204934+0000", 2024-04-26T04:32:25.346 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.26e-06 2024-04-26T04:32:25.346 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.346 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.346 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:25.346 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.205074+0000", 2024-04-26T04:32:25.346 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000140739 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.205117+0000", 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3102999999999997e-05 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.205146+0000", 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8750000000000001e-05 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.205280+0000", 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013341899999999999 2024-04-26T04:32:25.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.205288+0000", 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9389999999999997e-06 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206003+0000", 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00071516400000000001 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206092+0000", 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8799000000000001e-05 2024-04-26T04:32:25.348 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 32/30 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.209787+0000", 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.13548613800000001, 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016951100000000001, 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:25.349 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.209787+0000", 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.209787+0000", 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.209788+0000", 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7099999999999999e-06 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.350 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.209798+0000", 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2599999999999994e-06 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.209823+0000", 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5170000000000001e-05 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.209836+0000", 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3609e-05 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:25.351 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.209924+0000", 2024-04-26T04:32:25.352 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.7973000000000006e-05 2024-04-26T04:32:25.352 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.352 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.352 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:25.352 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.209956+0000", 2024-04-26T04:32:25.352 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1788999999999997e-05 2024-04-26T04:32:25.352 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.352 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:25.352 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.352 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.352 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.352 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 32/30 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.210075+0000", 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.135197978, 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00047873600000000001, 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210075+0000", 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:25.353 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210075+0000", 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210076+0000", 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2399999999999996e-07 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210081+0000", 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2360000000000004e-06 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.354 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210098+0000", 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6764000000000001e-05 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210107+0000", 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4800000000000007e-06 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210523+0000", 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00041591600000000001 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:25.355 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210554+0000", 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0416000000000002e-05 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 31/30 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.210396+0000", 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.134876517, 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00054738200000000001, 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:25.356 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210396+0000", 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210396+0000", 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210398+0000", 2024-04-26T04:32:25.357 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.48e-06 2024-04-26T04:32:25.358 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.358 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.358 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:25.358 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210407+0000", 2024-04-26T04:32:25.358 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.55e-06 2024-04-26T04:32:25.358 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.358 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.358 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:25.358 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210425+0000", 2024-04-26T04:32:25.358 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.753e-05 2024-04-26T04:32:25.358 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.358 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.358 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210436+0000", 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0693000000000001e-05 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210922+0000", 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000486269 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210944+0000", 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1860000000000001e-05 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.359 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.4s5 32/30 ECSubWrite(tid=1, reqid=client.4670.0:11, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.210549+0000", 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.134723278, 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001023714, 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:25.360 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210549+0000", 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210549+0000", 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210550+0000", 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7300000000000005e-07 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:25.361 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210586+0000", 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6199999999999999e-05 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210607+0000", 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0072999999999999e-05 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210618+0000", 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1463e-05 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.362 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210764+0000", 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014629199999999999 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210792+0000", 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7617e-05 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.211488+0000", 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00069570000000000005 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.363 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.211573+0000", 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5595999999999998e-05 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 31/30 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.210830+0000", 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.13444307699999999, 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0031424249999999999, 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:25.364 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210830+0000", 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210830+0000", 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210831+0000", 2024-04-26T04:32:25.365 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.02e-06 2024-04-26T04:32:25.366 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.366 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.366 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:25.366 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210838+0000", 2024-04-26T04:32:25.366 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1230000000000002e-06 2024-04-26T04:32:25.366 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.366 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.366 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:25.366 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210851+0000", 2024-04-26T04:32:25.366 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3502999999999999e-05 2024-04-26T04:32:25.366 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.366 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.366 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.210860+0000", 2024-04-26T04:32:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5799999999999992e-06 2024-04-26T04:32:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213929+0000", 2024-04-26T04:32:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0030688830000000001 2024-04-26T04:32:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213972+0000", 2024-04-26T04:32:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3316000000000003e-05 2024-04-26T04:32:25.367 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.368 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:25.368 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.368 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.368 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.368 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 32/30 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-04-26T04:32:25.368 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.213756+0000", 2024-04-26T04:32:25.368 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.13151676300000001, 2024-04-26T04:32:25.368 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00028166999999999998, 2024-04-26T04:32:25.368 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:25.368 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:25.368 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:25.368 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:25.368 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213756+0000", 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213756+0000", 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213758+0000", 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6670000000000001e-06 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.369 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213768+0000", 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.006e-05 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213795+0000", 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6789000000000001e-05 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213808+0000", 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3743e-05 2024-04-26T04:32:25.370 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.214016+0000", 2024-04-26T04:32:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00020758199999999999 2024-04-26T04:32:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.214038+0000", 2024-04-26T04:32:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1829000000000001e-05 2024-04-26T04:32:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.371 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 32/30 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.213837+0000", 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.131436161, 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.086405151999999999, 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213837+0000", 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.372 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213837+0000", 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213837+0000", 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7400000000000003e-07 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213841+0000", 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.393e-06 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.373 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213853+0000", 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2627e-05 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.213863+0000", 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4660000000000005e-06 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.300209+0000", 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.086346399000000004 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.374 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.300242+0000", 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2693e-05 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.as4 32/30 ECSubWrite(tid=0, reqid=client.4670.0:40, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.218637+0000", 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.126635317, 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10236036, 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:25.375 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:25.376 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:25.376 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:25.376 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.376 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:25.376 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.218637+0000", 2024-04-26T04:32:25.376 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.376 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.376 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.376 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:25.376 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.218637+0000", 2024-04-26T04:32:25.376 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.376 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.376 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.218639+0000", 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9700000000000002e-06 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.218900+0000", 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00026033499999999999 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.218927+0000", 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7708999999999998e-05 2024-04-26T04:32:25.377 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.319063+0000", 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10013590999999999 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.319278+0000", 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000214155 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.319306+0000", 2024-04-26T04:32:25.378 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8039999999999999e-05 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.320919+0000", 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016134019999999999 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.320998+0000", 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8838999999999995e-05 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.379 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.as4 32/30 ECSubWrite(tid=1, reqid=client.4670.0:41, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.319117+0000", 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.026155749999999998, 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001949746, 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.319117+0000", 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.380 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.381 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.381 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:25.381 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.319117+0000", 2024-04-26T04:32:25.381 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:25.381 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.381 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.381 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:25.381 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.319118+0000", 2024-04-26T04:32:25.381 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1000000000000001e-06 2024-04-26T04:32:25.381 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.381 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.381 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:25.381 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.319152+0000", 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3952999999999998e-05 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.319171+0000", 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9400000000000001e-05 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.319184+0000", 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2036e-05 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.382 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.320252+0000", 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0010680830000000001 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.320259+0000", 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6369999999999999e-06 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.321008+0000", 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00074897099999999997 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:25.383 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:25.384 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:25.384 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.321067+0000", 2024-04-26T04:32:25.384 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8566000000000001e-05 2024-04-26T04:32:25.384 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.384 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:25.384 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.384 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:25.384 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:25.384 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:25.385 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T04:32:25.386 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:25.383+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:25.410 INFO:tasks.rados.rados.0.smithi178.stdout:4: finishing write tid 1 to smithi17829308-4 2024-04-26T04:32:25.416 INFO:tasks.rados.rados.0.smithi178.stdout:14: finishing write tid 2 to smithi17829308-14 2024-04-26T04:32:25.443 INFO:tasks.rados.rados.0.smithi178.stdout:3: finishing write tid 1 to smithi17829308-3 2024-04-26T04:32:25.486 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:25.483+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:25.510 INFO:tasks.rados.rados.0.smithi178.stdout:8: finishing write tid 1 to smithi17829308-8 2024-04-26T04:32:25.510 INFO:tasks.rados.rados.0.smithi178.stdout:1: finishing write tid 1 to smithi17829308-1 2024-04-26T04:32:25.510 INFO:tasks.rados.rados.0.smithi178.stdout:2: finishing write tid 1 to smithi17829308-2 2024-04-26T04:32:25.515 INFO:tasks.rados.rados.0.smithi178.stdout:4: finishing write tid 2 to smithi17829308-4 2024-04-26T04:32:25.521 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:25.521 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:25.531 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:25.531 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:25.536 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T04:32:25.553 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:25.553 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:25.575 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:25.575 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:25.587 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:25.582+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:25.595 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:25.595 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:25.607 INFO:tasks.rados.rados.0.smithi178.stdout:7: finishing write tid 1 to smithi17829308-7 2024-04-26T04:32:25.613 INFO:tasks.rados.rados.0.smithi178.stdout:1: finishing write tid 2 to smithi17829308-1 2024-04-26T04:32:25.613 INFO:tasks.rados.rados.0.smithi178.stdout:8: finishing write tid 2 to smithi17829308-8 2024-04-26T04:32:25.613 INFO:tasks.rados.rados.0.smithi178.stdout:5: finishing write tid 1 to smithi17829308-5 2024-04-26T04:32:25.613 INFO:tasks.rados.rados.0.smithi178.stdout:2: finishing write tid 2 to smithi17829308-2 2024-04-26T04:32:25.613 INFO:tasks.rados.rados.0.smithi178.stdout:5: finishing write tid 2 to smithi17829308-5 2024-04-26T04:32:25.621 INFO:tasks.rados.rados.0.smithi178.stdout:14: finishing write tid 3 to smithi17829308-14 2024-04-26T04:32:25.621 INFO:tasks.rados.rados.0.smithi178.stdout:14: oid 14 updating version 0 to 2 2024-04-26T04:32:25.621 INFO:tasks.rados.rados.0.smithi178.stdout:14: oid 14 version 2 is already newer than 1 2024-04-26T04:32:25.621 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 14 v 2 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-04-26T04:32:25.621 INFO:tasks.rados.rados.0.smithi178.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-04-26T04:32:25.621 INFO:tasks.rados.rados.0.smithi178.stdout:14: done (15 left) 2024-04-26T04:32:25.621 INFO:tasks.rados.rados.0.smithi178.stdout:17: write initial oid 17 2024-04-26T04:32:25.621 INFO:tasks.rados.rados.0.smithi178.stdout:17: seq_num 16 ranges {0=278528} 2024-04-26T04:32:25.623 INFO:tasks.rados.rados.0.smithi178.stdout:17: writing smithi17829308-17 from 0 to 278528 tid 1 2024-04-26T04:32:25.623 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:25.688 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:25.684+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:25.702 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:25.702 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:25.706 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [2] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T04:32:25.707 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T04:32:25.707 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2024-04-26T04:32:25.707 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:32:25.713 INFO:tasks.rados.rados.0.smithi178.stdout:3: finishing write tid 2 to smithi17829308-3 2024-04-26T04:32:25.715 INFO:tasks.rados.rados.0.smithi178.stdout:6: finishing write tid 1 to smithi17829308-6 2024-04-26T04:32:25.715 INFO:tasks.rados.rados.0.smithi178.stdout:7: finishing write tid 2 to smithi17829308-7 2024-04-26T04:32:25.716 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T04:32:25.716 INFO:tasks.rados.rados.0.smithi178.stdout:6: finishing write tid 2 to smithi17829308-6 2024-04-26T04:32:25.717 INFO:tasks.rados.rados.0.smithi178.stdout:13: finishing write tid 1 to smithi17829308-13 2024-04-26T04:32:25.719 INFO:tasks.rados.rados.0.smithi178.stdout:12: finishing write tid 1 to smithi17829308-12 2024-04-26T04:32:25.720 INFO:tasks.rados.rados.0.smithi178.stdout:12: finishing write tid 2 to smithi17829308-12 2024-04-26T04:32:25.789 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:25.787+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:25.813 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:25.813 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:25.814 INFO:tasks.rados.rados.0.smithi178.stdout:13: finishing write tid 2 to smithi17829308-13 2024-04-26T04:32:25.817 INFO:tasks.rados.rados.0.smithi178.stdout:2: finishing write tid 3 to smithi17829308-2 2024-04-26T04:32:25.818 INFO:tasks.rados.rados.0.smithi178.stdout:2: oid 2 updating version 0 to 1 2024-04-26T04:32:25.818 INFO:tasks.rados.rados.0.smithi178.stdout:2: oid 2 updating version 1 to 2 2024-04-26T04:32:25.818 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 2 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-04-26T04:32:25.818 INFO:tasks.rados.rados.0.smithi178.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-04-26T04:32:25.818 INFO:tasks.rados.rados.0.smithi178.stdout:2: done (15 left) 2024-04-26T04:32:25.818 INFO:tasks.rados.rados.0.smithi178.stdout:18: write initial oid 18 2024-04-26T04:32:25.818 INFO:tasks.rados.rados.0.smithi178.stdout:18: seq_num 17 ranges {0=294912} 2024-04-26T04:32:25.820 INFO:tasks.rados.rados.0.smithi178.stdout:18: writing smithi17829308-18 from 0 to 294912 tid 1 2024-04-26T04:32:25.820 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:25.820 INFO:tasks.rados.rados.0.smithi178.stdout:8: finishing write tid 3 to smithi17829308-8 2024-04-26T04:32:25.820 INFO:tasks.rados.rados.0.smithi178.stdout:8: oid 8 updating version 0 to 1 2024-04-26T04:32:25.820 INFO:tasks.rados.rados.0.smithi178.stdout:8: oid 8 updating version 1 to 2 2024-04-26T04:32:25.820 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 8 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-26T04:32:25.820 INFO:tasks.rados.rados.0.smithi178.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-04-26T04:32:25.820 INFO:tasks.rados.rados.0.smithi178.stdout:8: done (15 left) 2024-04-26T04:32:25.820 INFO:tasks.rados.rados.0.smithi178.stdout:19: write initial oid 19 2024-04-26T04:32:25.820 INFO:tasks.rados.rados.0.smithi178.stdout:19: seq_num 18 ranges {0=311296} 2024-04-26T04:32:25.826 INFO:tasks.rados.rados.0.smithi178.stdout:19: writing smithi17829308-19 from 0 to 311296 tid 1 2024-04-26T04:32:25.826 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:25.826 INFO:tasks.rados.rados.0.smithi178.stdout:5: finishing write tid 3 to smithi17829308-5 2024-04-26T04:32:25.826 INFO:tasks.rados.rados.0.smithi178.stdout:5: oid 5 updating version 0 to 3 2024-04-26T04:32:25.826 INFO:tasks.rados.rados.0.smithi178.stdout:5: oid 5 updating version 3 to 4 2024-04-26T04:32:25.826 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 5 v 4 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-26T04:32:25.826 INFO:tasks.rados.rados.0.smithi178.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-04-26T04:32:25.826 INFO:tasks.rados.rados.0.smithi178.stdout:5: done (15 left) 2024-04-26T04:32:25.827 INFO:tasks.rados.rados.0.smithi178.stdout:20: write initial oid 20 2024-04-26T04:32:25.827 INFO:tasks.rados.rados.0.smithi178.stdout:20: seq_num 19 ranges {0=327680} 2024-04-26T04:32:25.831 INFO:tasks.rados.rados.0.smithi178.stdout:20: writing smithi17829308-20 from 0 to 327680 tid 1 2024-04-26T04:32:25.831 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:25.841 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:25.841 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:25.845 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:25.845 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:25.855 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T04:32:25.868 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:25.868 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:25.891 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:25.886+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:25.910 INFO:tasks.rados.rados.0.smithi178.stdout:10: finishing write tid 1 to smithi17829308-10 2024-04-26T04:32:25.917 INFO:tasks.rados.rados.0.smithi178.stdout:10: finishing write tid 2 to smithi17829308-10 2024-04-26T04:32:25.917 INFO:tasks.rados.rados.0.smithi178.stdout:12: finishing write tid 3 to smithi17829308-12 2024-04-26T04:32:25.917 INFO:tasks.rados.rados.0.smithi178.stdout:12: oid 12 updating version 0 to 5 2024-04-26T04:32:25.917 INFO:tasks.rados.rados.0.smithi178.stdout:12: oid 12 updating version 5 to 6 2024-04-26T04:32:25.917 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 12 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-04-26T04:32:25.917 INFO:tasks.rados.rados.0.smithi178.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-04-26T04:32:25.917 INFO:tasks.rados.rados.0.smithi178.stdout:3: finishing write tid 3 to smithi17829308-3 2024-04-26T04:32:25.917 INFO:tasks.rados.rados.0.smithi178.stdout:3: oid 3 updating version 0 to 2 2024-04-26T04:32:25.917 INFO:tasks.rados.rados.0.smithi178.stdout:3: oid 3 version 2 is already newer than 1 2024-04-26T04:32:25.917 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 3 v 2 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-04-26T04:32:25.918 INFO:tasks.rados.rados.0.smithi178.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-04-26T04:32:25.918 INFO:tasks.rados.rados.0.smithi178.stdout:6: finishing write tid 3 to smithi17829308-6 2024-04-26T04:32:25.918 INFO:tasks.rados.rados.0.smithi178.stdout:6: oid 6 updating version 0 to 4 2024-04-26T04:32:25.918 INFO:tasks.rados.rados.0.smithi178.stdout:6: oid 6 version 4 is already newer than 3 2024-04-26T04:32:25.918 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 6 v 4 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-04-26T04:32:25.918 INFO:tasks.rados.rados.0.smithi178.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-04-26T04:32:25.918 INFO:tasks.rados.rados.0.smithi178.stdout:3: done (15 left) 2024-04-26T04:32:25.918 INFO:tasks.rados.rados.0.smithi178.stdout:6: done (14 left) 2024-04-26T04:32:25.918 INFO:tasks.rados.rados.0.smithi178.stdout:12: done (13 left) 2024-04-26T04:32:25.918 INFO:tasks.rados.rados.0.smithi178.stdout:21: write initial oid 21 2024-04-26T04:32:25.918 INFO:tasks.rados.rados.0.smithi178.stdout:21: seq_num 20 ranges {0=344064} 2024-04-26T04:32:25.920 INFO:tasks.rados.rados.0.smithi178.stdout:21: writing smithi17829308-21 from 0 to 344064 tid 1 2024-04-26T04:32:25.920 INFO:tasks.rados.rados.0.smithi178.stdout:22: write initial oid 22 2024-04-26T04:32:25.920 INFO:tasks.rados.rados.0.smithi178.stdout:22: seq_num 21 ranges {0=360448} 2024-04-26T04:32:25.922 INFO:tasks.rados.rados.0.smithi178.stdout:22: writing smithi17829308-22 from 0 to 360448 tid 1 2024-04-26T04:32:25.923 INFO:tasks.rados.rados.0.smithi178.stdout:23: write initial oid 23 2024-04-26T04:32:25.923 INFO:tasks.rados.rados.0.smithi178.stdout:23: seq_num 22 ranges {0=376832} 2024-04-26T04:32:25.926 INFO:tasks.rados.rados.0.smithi178.stdout:23: writing smithi17829308-23 from 0 to 376832 tid 1 2024-04-26T04:32:25.926 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:25.926 INFO:tasks.rados.rados.0.smithi178.stdout:16: finishing write tid 1 to smithi17829308-16 2024-04-26T04:32:25.926 INFO:tasks.rados.rados.0.smithi178.stdout:16: finishing write tid 2 to smithi17829308-16 2024-04-26T04:32:25.926 INFO:tasks.rados.rados.0.smithi178.stdout:1: finishing write tid 3 to smithi17829308-1 2024-04-26T04:32:25.926 INFO:tasks.rados.rados.0.smithi178.stdout:1: oid 1 updating version 0 to 2 2024-04-26T04:32:25.926 INFO:tasks.rados.rados.0.smithi178.stdout:1: oid 1 version 2 is already newer than 1 2024-04-26T04:32:25.926 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 1 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-04-26T04:32:25.927 INFO:tasks.rados.rados.0.smithi178.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-04-26T04:32:25.927 INFO:tasks.rados.rados.0.smithi178.stdout:1: done (15 left) 2024-04-26T04:32:25.927 INFO:tasks.rados.rados.0.smithi178.stdout:24: write initial oid 24 2024-04-26T04:32:25.927 INFO:tasks.rados.rados.0.smithi178.stdout:24: seq_num 23 ranges {0=393216} 2024-04-26T04:32:25.928 INFO:tasks.rados.rados.0.smithi178.stdout:24: writing smithi17829308-24 from 0 to 393216 tid 1 2024-04-26T04:32:25.929 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:25.940 INFO:tasks.rados.rados.0.smithi178.stdout:20: finishing write tid 1 to smithi17829308-20 2024-04-26T04:32:25.949 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:25.949 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:25.980 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:25.980 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:25.991 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:25.991+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:26.011 INFO:tasks.rados.rados.0.smithi178.stdout:9: finishing write tid 1 to smithi17829308-9 2024-04-26T04:32:26.016 INFO:tasks.rados.rados.0.smithi178.stdout:9: finishing write tid 2 to smithi17829308-9 2024-04-26T04:32:26.019 INFO:tasks.rados.rados.0.smithi178.stdout:13: finishing write tid 3 to smithi17829308-13 2024-04-26T04:32:26.019 INFO:tasks.rados.rados.0.smithi178.stdout:13: oid 13 updating version 0 to 5 2024-04-26T04:32:26.019 INFO:tasks.rados.rados.0.smithi178.stdout:13: oid 13 updating version 5 to 6 2024-04-26T04:32:26.019 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 13 v 6 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-04-26T04:32:26.019 INFO:tasks.rados.rados.0.smithi178.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-04-26T04:32:26.019 INFO:tasks.rados.rados.0.smithi178.stdout:13: done (15 left) 2024-04-26T04:32:26.019 INFO:tasks.rados.rados.0.smithi178.stdout:25: write initial oid 25 2024-04-26T04:32:26.019 INFO:tasks.rados.rados.0.smithi178.stdout:25: seq_num 24 ranges {0=409600} 2024-04-26T04:32:26.027 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:26.027 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:26.027 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:26.028 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:32:26.029 INFO:tasks.rados.rados.0.smithi178.stdout:25: writing smithi17829308-25 from 0 to 409600 tid 1 2024-04-26T04:32:26.029 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.029 INFO:tasks.rados.rados.0.smithi178.stdout:7: finishing write tid 3 to smithi17829308-7 2024-04-26T04:32:26.029 INFO:tasks.rados.rados.0.smithi178.stdout:7: oid 7 updating version 0 to 1 2024-04-26T04:32:26.029 INFO:tasks.rados.rados.0.smithi178.stdout:7: oid 7 updating version 1 to 2 2024-04-26T04:32:26.029 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 7 v 2 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-04-26T04:32:26.029 INFO:tasks.rados.rados.0.smithi178.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-04-26T04:32:26.030 INFO:tasks.rados.rados.0.smithi178.stdout:17: finishing write tid 1 to smithi17829308-17 2024-04-26T04:32:26.030 INFO:tasks.rados.rados.0.smithi178.stdout:17: finishing write tid 2 to smithi17829308-17 2024-04-26T04:32:26.030 INFO:tasks.rados.rados.0.smithi178.stdout:11: finishing write tid 1 to smithi17829308-11 2024-04-26T04:32:26.030 INFO:tasks.rados.rados.0.smithi178.stdout:11: finishing write tid 2 to smithi17829308-11 2024-04-26T04:32:26.030 INFO:tasks.rados.rados.0.smithi178.stdout:15: finishing write tid 1 to smithi17829308-15 2024-04-26T04:32:26.030 INFO:tasks.rados.rados.0.smithi178.stdout:15: finishing write tid 2 to smithi17829308-15 2024-04-26T04:32:26.030 INFO:tasks.rados.rados.0.smithi178.stdout:7: done (15 left) 2024-04-26T04:32:26.030 INFO:tasks.rados.rados.0.smithi178.stdout:26: write initial oid 26 2024-04-26T04:32:26.030 INFO:tasks.rados.rados.0.smithi178.stdout:26: seq_num 25 ranges {0=425984} 2024-04-26T04:32:26.035 INFO:tasks.rados.rados.0.smithi178.stdout:26: writing smithi17829308-26 from 0 to 425984 tid 1 2024-04-26T04:32:26.035 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.039 INFO:tasks.rados.rados.0.smithi178.stdout:18: finishing write tid 1 to smithi17829308-18 2024-04-26T04:32:26.040 INFO:tasks.rados.rados.0.smithi178.stdout:18: finishing write tid 2 to smithi17829308-18 2024-04-26T04:32:26.042 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T04:32:26.042 INFO:tasks.rados.rados.0.smithi178.stdout:20: finishing write tid 2 to smithi17829308-20 2024-04-26T04:32:26.046 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-04-26T04:32:25.328409+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":107,"num_write":182,"num_write_kb":2941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":88,"acting":88,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":15,"num_per_pool_osds":15,"num_per_pool_omap_osds":11,"kb":1415577600,"kb_used":419572,"kb_used_data":5444,"kb_used_omap":23,"kb_used_meta":413416,"kb_avail":1415158028,"statfs":{"total":1449551462400,"available":1449121820672,"internally_reserved":0,"allocated":5574656,"data_stored":2895255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23926,"internal_metadata":423338634},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001169"},"pg_stats":[{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T04:32:23.580179+0000","last_change":"2024-04-26T04:32:23.579102+0000","last_active":"2024-04-26T04:32:23.580179+0000","last_peered":"2024-04-26T04:32:23.580179+0000","last_clean":"2024-04-26T04:32:23.580179+0000","last_became_active":"2024-04-26T04:32:23.378967+0000","last_became_peered":"2024-04-26T04:32:23.378967+0000","last_unstale":"2024-04-26T04:32:23.580179+0000","last_undegraded":"2024-04-26T04:32:23.580179+0000","last_fullsized":"2024-04-26T04:32:23.580179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:23.216391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,14,13,8,10,0],"acting":[11,1,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T04:32:22.154164+0000","last_change":"2024-04-26T04:32:22.154164+0000","last_active":"2024-04-26T04:32:22.154164+0000","last_peered":"2024-04-26T04:32:22.154164+0000","last_clean":"2024-04-26T04:32:22.154164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T04:32:22.154164+0000","last_undegraded":"2024-04-26T04:32:22.154164+0000","last_fullsized":"2024-04-26T04:32:22.154164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T04:32:23.694842+0000","last_change":"2024-04-26T04:32:23.594045+0000","last_active":"2024-04-26T04:32:23.694842+0000","last_peered":"2024-04-26T04:32:23.694842+0000","last_clean":"2024-04-26T04:32:23.694842+0000","last_became_active":"2024-04-26T04:32:23.392585+0000","last_became_peered":"2024-04-26T04:32:23.392585+0000","last_unstale":"2024-04-26T04:32:23.694842+0000","last_undegraded":"2024-04-26T04:32:23.694842+0000","last_fullsized":"2024-04-26T04:32:23.694842+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:24.179564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T04:32:23.580551+0000","last_change":"2024-04-26T04:32:23.579580+0000","last_active":"2024-04-26T04:32:23.580551+0000","last_peered":"2024-04-26T04:32:23.580551+0000","last_clean":"2024-04-26T04:32:23.580551+0000","last_became_active":"2024-04-26T04:32:23.577814+0000","last_became_peered":"2024-04-26T04:32:23.577814+0000","last_unstale":"2024-04-26T04:32:23.580551+0000","last_undegraded":"2024-04-26T04:32:23.580551+0000","last_fullsized":"2024-04-26T04:32:23.580551+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:23.160229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-26T04:32:22.184029+0000","last_change":"2024-04-26T04:32:22.184029+0000","last_active":"2024-04-26T04:32:22.154164+0000","last_peered":"2024-04-26T04:32:22.154164+0000","last_clean":"2024-04-26T04:32:22.154164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T04:32:22.184029+0000","last_undegraded":"2024-04-26T04:32:22.184029+0000","last_fullsized":"2024-04-26T04:32:22.184029+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T04:32:23.313188+0000","last_change":"2024-04-26T04:32:23.312351+0000","last_active":"2024-04-26T04:32:23.313188+0000","last_peered":"2024-04-26T04:32:23.313188+0000","last_clean":"2024-04-26T04:32:23.313188+0000","last_became_active":"2024-04-26T04:32:23.310795+0000","last_became_peered":"2024-04-26T04:32:23.310795+0000","last_unstale":"2024-04-26T04:32:23.313188+0000","last_undegraded":"2024-04-26T04:32:23.313188+0000","last_fullsized":"2024-04-26T04:32:23.313188+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:32.579899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T04:32:22.154164+0000","last_change":"2024-04-26T04:32:22.154164+0000","last_active":"2024-04-26T04:32:22.154164+0000","last_peered":"2024-04-26T04:32:22.154164+0000","last_clean":"2024-04-26T04:32:22.154164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T04:32:22.154164+0000","last_undegraded":"2024-04-26T04:32:22.154164+0000","last_fullsized":"2024-04-26T04:32:22.154164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T04:32:22.154164+0000","last_change":"2024-04-26T04:32:22.154164+0000","last_active":"2024-04-26T04:32:22.154164+0000","last_peered":"2024-04-26T04:32:22.154164+0000","last_clean":"2024-04-26T04:32:22.154164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T04:32:22.154164+0000","last_undegraded":"2024-04-26T04:32:22.154164+0000","last_fullsized":"2024-04-26T04:32:22.154164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"27'139","reported_seq":187,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:20.321023+0000","last_change":"2024-04-26T04:32:10.731528+0000","last_active":"2024-04-26T04:32:20.321023+0000","last_peered":"2024-04-26T04:32:20.321023+0000","last_clean":"2024-04-26T04:32:20.321023+0000","last_became_active":"2024-04-26T04:32:10.729056+0000","last_became_peered":"2024-04-26T04:32:10.729056+0000","last_unstale":"2024-04-26T04:32:20.321023+0000","last_undegraded":"2024-04-26T04:32:20.321023+0000","last_fullsized":"2024-04-26T04:32:20.321023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:32:08.365921+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:10.391321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":107,"num_write":182,"num_write_kb":2941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.505930+0000","last_change":"2024-04-26T04:32:12.505010+0000","last_active":"2024-04-26T04:32:13.505930+0000","last_peered":"2024-04-26T04:32:13.505930+0000","last_clean":"2024-04-26T04:32:13.505930+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:32:13.505930+0000","last_undegraded":"2024-04-26T04:32:13.505930+0000","last_fullsized":"2024-04-26T04:32:13.505930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.374592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T04:32:22.154164+0000","last_change":"2024-04-26T04:32:22.154164+0000","last_active":"2024-04-26T04:32:22.154164+0000","last_peered":"2024-04-26T04:32:22.154164+0000","last_clean":"2024-04-26T04:32:22.154164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T04:32:22.154164+0000","last_undegraded":"2024-04-26T04:32:22.154164+0000","last_fullsized":"2024-04-26T04:32:22.154164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T04:32:23.290225+0000","last_change":"2024-04-26T04:32:12.603501+0000","last_active":"2024-04-26T04:32:23.290225+0000","last_peered":"2024-04-26T04:32:23.290225+0000","last_clean":"2024-04-26T04:32:23.290225+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:32:23.290225+0000","last_undegraded":"2024-04-26T04:32:23.290225+0000","last_fullsized":"2024-04-26T04:32:23.290225+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.945207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-26T04:32:22.184013+0000","last_change":"2024-04-26T04:32:22.184013+0000","last_active":"2024-04-26T04:32:22.154164+0000","last_peered":"2024-04-26T04:32:22.154164+0000","last_clean":"2024-04-26T04:32:22.154164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T04:32:22.184013+0000","last_undegraded":"2024-04-26T04:32:22.184013+0000","last_fullsized":"2024-04-26T04:32:22.184013+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,12,13],"acting":[4,11,9,5,10,12,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.512246+0000","last_change":"2024-04-26T04:32:12.505739+0000","last_active":"2024-04-26T04:32:13.512246+0000","last_peered":"2024-04-26T04:32:13.512246+0000","last_clean":"2024-04-26T04:32:13.512246+0000","last_became_active":"2024-04-26T04:32:12.504908+0000","last_became_peered":"2024-04-26T04:32:12.504908+0000","last_unstale":"2024-04-26T04:32:13.512246+0000","last_undegraded":"2024-04-26T04:32:13.512246+0000","last_fullsized":"2024-04-26T04:32:13.512246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:41.080750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T04:32:22.154164+0000","last_change":"2024-04-26T04:32:22.154164+0000","last_active":"2024-04-26T04:32:22.154164+0000","last_peered":"2024-04-26T04:32:22.154164+0000","last_clean":"2024-04-26T04:32:22.154164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T04:32:22.154164+0000","last_undegraded":"2024-04-26T04:32:22.154164+0000","last_fullsized":"2024-04-26T04:32:22.154164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-26T04:32:23.192186+0000","last_change":"2024-04-26T04:32:12.798464+0000","last_active":"2024-04-26T04:32:23.192186+0000","last_peered":"2024-04-26T04:32:23.192186+0000","last_clean":"2024-04-26T04:32:23.192186+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:32:23.192186+0000","last_undegraded":"2024-04-26T04:32:23.192186+0000","last_fullsized":"2024-04-26T04:32:23.192186+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.684262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T04:32:22.154164+0000","last_change":"2024-04-26T04:32:22.154164+0000","last_active":"2024-04-26T04:32:22.154164+0000","last_peered":"2024-04-26T04:32:22.154164+0000","last_clean":"2024-04-26T04:32:22.154164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T04:32:22.154164+0000","last_undegraded":"2024-04-26T04:32:22.154164+0000","last_fullsized":"2024-04-26T04:32:22.154164+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-26T04:32:22.183359+0000","last_change":"2024-04-26T04:32:22.183359+0000","last_active":"2024-04-26T04:32:22.154164+0000","last_peered":"2024-04-26T04:32:22.154164+0000","last_clean":"2024-04-26T04:32:22.154164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T04:32:22.183359+0000","last_undegraded":"2024-04-26T04:32:22.183359+0000","last_fullsized":"2024-04-26T04:32:22.183359+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-26T04:32:22.185550+0000","last_change":"2024-04-26T04:32:13.108823+0000","last_active":"2024-04-26T04:32:22.185550+0000","last_peered":"2024-04-26T04:32:22.185550+0000","last_clean":"2024-04-26T04:32:22.185550+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:32:22.185550+0000","last_undegraded":"2024-04-26T04:32:22.185550+0000","last_fullsized":"2024-04-26T04:32:22.185550+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-26T04:32:22.181630+0000","last_change":"2024-04-26T04:32:22.181630+0000","last_active":"2024-04-26T04:32:22.154164+0000","last_peered":"2024-04-26T04:32:22.154164+0000","last_clean":"2024-04-26T04:32:22.154164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T04:32:22.181630+0000","last_undegraded":"2024-04-26T04:32:22.181630+0000","last_fullsized":"2024-04-26T04:32:22.181630+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,4,1,0,14,13],"acting":[8,9,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-26T04:32:13.723527+0000","last_change":"2024-04-26T04:32:12.504400+0000","last_active":"2024-04-26T04:32:13.723527+0000","last_peered":"2024-04-26T04:32:13.723527+0000","last_clean":"2024-04-26T04:32:13.723527+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:32:13.723527+0000","last_undegraded":"2024-04-26T04:32:13.723527+0000","last_fullsized":"2024-04-26T04:32:13.723527+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.381954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-26T04:32:22.183841+0000","last_change":"2024-04-26T04:32:22.183841+0000","last_active":"2024-04-26T04:32:22.154164+0000","last_peered":"2024-04-26T04:32:22.154164+0000","last_clean":"2024-04-26T04:32:22.154164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T04:32:22.183841+0000","last_undegraded":"2024-04-26T04:32:22.183841+0000","last_fullsized":"2024-04-26T04:32:22.183841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-26T04:32:22.283844+0000","last_change":"2024-04-26T04:32:12.697838+0000","last_active":"2024-04-26T04:32:22.283844+0000","last_peered":"2024-04-26T04:32:22.283844+0000","last_clean":"2024-04-26T04:32:22.283844+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:32:22.283844+0000","last_undegraded":"2024-04-26T04:32:22.283844+0000","last_fullsized":"2024-04-26T04:32:22.283844+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.961816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-04-26T04:32:22.177297+0000","last_change":"2024-04-26T04:32:22.177297+0000","last_active":"2024-04-26T04:32:22.154164+0000","last_peered":"2024-04-26T04:32:22.154164+0000","last_clean":"2024-04-26T04:32:22.154164+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T04:32:22.177297+0000","last_undegraded":"2024-04-26T04:32:22.177297+0000","last_fullsized":"2024-04-26T04:32:22.177297+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-04-26T04:32:22.185208+0000","last_change":"2024-04-26T04:32:12.803651+0000","last_active":"2024-04-26T04:32:22.185208+0000","last_peered":"2024-04-26T04:32:22.185208+0000","last_clean":"2024-04-26T04:32:22.185208+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:32:22.185208+0000","last_undegraded":"2024-04-26T04:32:22.185208+0000","last_fullsized":"2024-04-26T04:32:22.185208+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:26.614652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":70,"acting":70,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":107,"num_write":182,"num_write_kb":2941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1474560,"data_stored":1463904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247815,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247814,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":749568,"data_stored":577516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313222,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":141171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,7,8,10,11,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313222,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":141171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,6,7,8,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313222,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":141171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[3,4,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313222,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":119769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313222,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":130906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280518,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":876544,"data_stored":680413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[2,4,6,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":344064,"data_stored":151485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,3,4,6,7,8,9,10,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247815,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,4,6,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247813,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":344064,"data_stored":151485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247815,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":278528,"data_stored":110028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:32:26.067 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:26.067 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:26.092 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:26.088+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:26.101 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:26.102 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:26.122 INFO:tasks.rados.rados.0.smithi178.stdout:4: finishing write tid 3 to smithi17829308-4 2024-04-26T04:32:26.122 INFO:tasks.rados.rados.0.smithi178.stdout:4: oid 4 updating version 0 to 2 2024-04-26T04:32:26.122 INFO:tasks.rados.rados.0.smithi178.stdout:4: oid 4 version 2 is already newer than 1 2024-04-26T04:32:26.122 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-04-26T04:32:26.122 INFO:tasks.rados.rados.0.smithi178.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-04-26T04:32:26.122 INFO:tasks.rados.rados.0.smithi178.stdout:4: done (15 left) 2024-04-26T04:32:26.122 INFO:tasks.rados.rados.0.smithi178.stdout:27: write initial oid 27 2024-04-26T04:32:26.122 INFO:tasks.rados.rados.0.smithi178.stdout:27: seq_num 26 ranges {0=442368} 2024-04-26T04:32:26.129 INFO:tasks.rados.rados.0.smithi178.stdout:27: writing smithi17829308-27 from 0 to 442368 tid 1 2024-04-26T04:32:26.129 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.130 INFO:tasks.rados.rados.0.smithi178.stdout:10: finishing write tid 3 to smithi17829308-10 2024-04-26T04:32:26.130 INFO:tasks.rados.rados.0.smithi178.stdout:10: oid 10 updating version 0 to 3 2024-04-26T04:32:26.130 INFO:tasks.rados.rados.0.smithi178.stdout:10: oid 10 updating version 3 to 4 2024-04-26T04:32:26.130 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 10 v 4 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-04-26T04:32:26.130 INFO:tasks.rados.rados.0.smithi178.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-04-26T04:32:26.130 INFO:tasks.rados.rados.0.smithi178.stdout:17: finishing write tid 3 to smithi17829308-17 2024-04-26T04:32:26.130 INFO:tasks.rados.rados.0.smithi178.stdout:17: oid 17 updating version 0 to 4 2024-04-26T04:32:26.130 INFO:tasks.rados.rados.0.smithi178.stdout:17: oid 17 version 4 is already newer than 3 2024-04-26T04:32:26.130 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 17 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-04-26T04:32:26.130 INFO:tasks.rados.rados.0.smithi178.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-04-26T04:32:26.130 INFO:tasks.rados.rados.0.smithi178.stdout:11: finishing write tid 3 to smithi17829308-11 2024-04-26T04:32:26.130 INFO:tasks.rados.rados.0.smithi178.stdout:11: oid 11 updating version 0 to 1 2024-04-26T04:32:26.130 INFO:tasks.rados.rados.0.smithi178.stdout:11: oid 11 updating version 1 to 2 2024-04-26T04:32:26.131 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 11 v 2 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-26T04:32:26.131 INFO:tasks.rados.rados.0.smithi178.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-04-26T04:32:26.131 INFO:tasks.rados.rados.0.smithi178.stdout:10: done (15 left) 2024-04-26T04:32:26.131 INFO:tasks.rados.rados.0.smithi178.stdout:11: done (14 left) 2024-04-26T04:32:26.131 INFO:tasks.rados.rados.0.smithi178.stdout:17: done (13 left) 2024-04-26T04:32:26.131 INFO:tasks.rados.rados.0.smithi178.stdout:28: write initial oid 28 2024-04-26T04:32:26.131 INFO:tasks.rados.rados.0.smithi178.stdout:28: seq_num 27 ranges {0=458752} 2024-04-26T04:32:26.133 INFO:tasks.rados.rados.0.smithi178.stdout:28: writing smithi17829308-28 from 0 to 458752 tid 1 2024-04-26T04:32:26.133 INFO:tasks.rados.rados.0.smithi178.stdout:29: write initial oid 29 2024-04-26T04:32:26.133 INFO:tasks.rados.rados.0.smithi178.stdout:29: seq_num 28 ranges {0=475136} 2024-04-26T04:32:26.137 INFO:tasks.rados.rados.0.smithi178.stdout:29: writing smithi17829308-29 from 0 to 475136 tid 1 2024-04-26T04:32:26.138 INFO:tasks.rados.rados.0.smithi178.stdout:30: write initial oid 30 2024-04-26T04:32:26.138 INFO:tasks.rados.rados.0.smithi178.stdout:30: seq_num 29 ranges {0=491520} 2024-04-26T04:32:26.147 INFO:tasks.rados.rados.0.smithi178.stdout:30: writing smithi17829308-30 from 0 to 491520 tid 1 2024-04-26T04:32:26.147 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.166 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:26.166 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:26.177 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:26.177 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:26.181 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T04:32:26.206 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-04-26T04:32:26.224 INFO:tasks.rados.rados.0.smithi178.stdout:16: finishing write tid 3 to smithi17829308-16 2024-04-26T04:32:26.224 INFO:tasks.rados.rados.0.smithi178.stdout:16: oid 16 updating version 0 to 5 2024-04-26T04:32:26.225 INFO:tasks.rados.rados.0.smithi178.stdout:16: oid 16 updating version 5 to 6 2024-04-26T04:32:26.225 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 16 v 6 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-04-26T04:32:26.225 INFO:tasks.rados.rados.0.smithi178.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-04-26T04:32:26.225 INFO:tasks.rados.rados.0.smithi178.stdout:16: done (15 left) 2024-04-26T04:32:26.225 INFO:tasks.rados.rados.0.smithi178.stdout:31: write initial oid 31 2024-04-26T04:32:26.226 INFO:tasks.rados.rados.0.smithi178.stdout:31: seq_num 30 ranges {0=507904} 2024-04-26T04:32:26.226 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:26.223+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:26.228 INFO:tasks.rados.rados.0.smithi178.stdout:31: writing smithi17829308-31 from 0 to 507904 tid 1 2024-04-26T04:32:26.228 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.229 INFO:tasks.rados.rados.0.smithi178.stdout:9: finishing write tid 3 to smithi17829308-9 2024-04-26T04:32:26.229 INFO:tasks.rados.rados.0.smithi178.stdout:9: oid 9 updating version 0 to 3 2024-04-26T04:32:26.229 INFO:tasks.rados.rados.0.smithi178.stdout:9: oid 9 updating version 3 to 4 2024-04-26T04:32:26.229 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 9 v 4 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-04-26T04:32:26.229 INFO:tasks.rados.rados.0.smithi178.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-04-26T04:32:26.229 INFO:tasks.rados.rados.0.smithi178.stdout:9: done (15 left) 2024-04-26T04:32:26.230 INFO:tasks.rados.rados.0.smithi178.stdout:32: write initial oid 32 2024-04-26T04:32:26.230 INFO:tasks.rados.rados.0.smithi178.stdout:32: seq_num 31 ranges {0=524288} 2024-04-26T04:32:26.235 INFO:tasks.rados.rados.0.smithi178.stdout:32: writing smithi17829308-32 from 0 to 524288 tid 1 2024-04-26T04:32:26.235 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.235 INFO:tasks.rados.rados.0.smithi178.stdout:15: finishing write tid 3 to smithi17829308-15 2024-04-26T04:32:26.235 INFO:tasks.rados.rados.0.smithi178.stdout:15: oid 15 updating version 0 to 3 2024-04-26T04:32:26.235 INFO:tasks.rados.rados.0.smithi178.stdout:15: oid 15 updating version 3 to 4 2024-04-26T04:32:26.236 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 15 v 4 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-04-26T04:32:26.236 INFO:tasks.rados.rados.0.smithi178.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-04-26T04:32:26.236 INFO:tasks.rados.rados.0.smithi178.stdout:15: done (15 left) 2024-04-26T04:32:26.236 INFO:tasks.rados.rados.0.smithi178.stdout:33: write initial oid 33 2024-04-26T04:32:26.236 INFO:tasks.rados.rados.0.smithi178.stdout:33: seq_num 32 ranges {0=540672} 2024-04-26T04:32:26.236 INFO:tasks.rados.rados.0.smithi178.stdout:33: writing smithi17829308-33 from 0 to 540672 tid 1 2024-04-26T04:32:26.236 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.240 INFO:tasks.rados.rados.0.smithi178.stdout:32: finishing write tid 1 to smithi17829308-32 2024-04-26T04:32:26.246 INFO:tasks.rados.rados.0.smithi178.stdout:20: finishing write tid 3 to smithi17829308-20 2024-04-26T04:32:26.246 INFO:tasks.rados.rados.0.smithi178.stdout:20: oid 20 updating version 0 to 1 2024-04-26T04:32:26.247 INFO:tasks.rados.rados.0.smithi178.stdout:20: oid 20 updating version 1 to 2 2024-04-26T04:32:26.247 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 20 v 2 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-04-26T04:32:26.247 INFO:tasks.rados.rados.0.smithi178.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-04-26T04:32:26.247 INFO:tasks.rados.rados.0.smithi178.stdout:20: done (15 left) 2024-04-26T04:32:26.247 INFO:tasks.rados.rados.0.smithi178.stdout:34: write initial oid 34 2024-04-26T04:32:26.247 INFO:tasks.rados.rados.0.smithi178.stdout:34: seq_num 33 ranges {0=557056} 2024-04-26T04:32:26.250 INFO:tasks.rados.rados.0.smithi178.stdout:34: writing smithi17829308-34 from 0 to 557056 tid 1 2024-04-26T04:32:26.250 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.294 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:26.292+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:26.305 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:26.305 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:26.319 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T04:32:26.322 INFO:tasks.rados.rados.0.smithi178.stdout:23: finishing write tid 1 to smithi17829308-23 2024-04-26T04:32:26.329 INFO:tasks.rados.rados.0.smithi178.stdout:22: finishing write tid 1 to smithi17829308-22 2024-04-26T04:32:26.330 INFO:tasks.rados.rados.0.smithi178.stdout:22: finishing write tid 2 to smithi17829308-22 2024-04-26T04:32:26.396 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:26.392+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:26.423 INFO:tasks.rados.rados.0.smithi178.stdout:23: finishing write tid 2 to smithi17829308-23 2024-04-26T04:32:26.429 INFO:tasks.rados.rados.0.smithi178.stdout:19: finishing write tid 1 to smithi17829308-19 2024-04-26T04:32:26.438 INFO:tasks.rados.rados.0.smithi178.stdout:28: finishing write tid 1 to smithi17829308-28 2024-04-26T04:32:26.439 INFO:tasks.rados.rados.0.smithi178.stdout:28: finishing write tid 2 to smithi17829308-28 2024-04-26T04:32:26.446 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:26.446 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:26.449 INFO:tasks.rados.rados.0.smithi178.stdout:27: finishing write tid 1 to smithi17829308-27 2024-04-26T04:32:26.449 INFO:tasks.rados.rados.0.smithi178.stdout:21: finishing write tid 1 to smithi17829308-21 2024-04-26T04:32:26.449 INFO:tasks.rados.rados.0.smithi178.stdout:25: finishing write tid 1 to smithi17829308-25 2024-04-26T04:32:26.449 INFO:tasks.rados.rados.0.smithi178.stdout:18: finishing write tid 3 to smithi17829308-18 2024-04-26T04:32:26.449 INFO:tasks.rados.rados.0.smithi178.stdout:18: oid 18 updating version 0 to 4 2024-04-26T04:32:26.449 INFO:tasks.rados.rados.0.smithi178.stdout:18: oid 18 version 4 is already newer than 3 2024-04-26T04:32:26.449 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 18 v 4 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-04-26T04:32:26.449 INFO:tasks.rados.rados.0.smithi178.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-04-26T04:32:26.449 INFO:tasks.rados.rados.0.smithi178.stdout:18: done (15 left) 2024-04-26T04:32:26.449 INFO:tasks.rados.rados.0.smithi178.stdout:35: write initial oid 35 2024-04-26T04:32:26.449 INFO:tasks.rados.rados.0.smithi178.stdout:35: seq_num 34 ranges {0=573440} 2024-04-26T04:32:26.458 INFO:tasks.rados.rados.0.smithi178.stdout:35: writing smithi17829308-35 from 0 to 573440 tid 1 2024-04-26T04:32:26.458 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.462 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T04:32:26.496 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:26.492+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:26.530 INFO:tasks.rados.rados.0.smithi178.stdout:23: finishing write tid 3 to smithi17829308-23 2024-04-26T04:32:26.530 INFO:tasks.rados.rados.0.smithi178.stdout:23: oid 23 updating version 0 to 3 2024-04-26T04:32:26.530 INFO:tasks.rados.rados.0.smithi178.stdout:23: oid 23 updating version 3 to 4 2024-04-26T04:32:26.530 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 23 v 4 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-04-26T04:32:26.531 INFO:tasks.rados.rados.0.smithi178.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-04-26T04:32:26.531 INFO:tasks.rados.rados.0.smithi178.stdout:23: done (15 left) 2024-04-26T04:32:26.531 INFO:tasks.rados.rados.0.smithi178.stdout:36: write initial oid 36 2024-04-26T04:32:26.531 INFO:tasks.rados.rados.0.smithi178.stdout:36: seq_num 35 ranges {0=589824} 2024-04-26T04:32:26.535 INFO:tasks.rados.rados.0.smithi178.stdout:36: writing smithi17829308-36 from 0 to 589824 tid 1 2024-04-26T04:32:26.535 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.540 INFO:tasks.rados.rados.0.smithi178.stdout:32: finishing write tid 2 to smithi17829308-32 2024-04-26T04:32:26.542 INFO:tasks.rados.rados.0.smithi178.stdout:28: finishing write tid 3 to smithi17829308-28 2024-04-26T04:32:26.542 INFO:tasks.rados.rados.0.smithi178.stdout:28: oid 28 updating version 0 to 1 2024-04-26T04:32:26.542 INFO:tasks.rados.rados.0.smithi178.stdout:28: oid 28 updating version 1 to 2 2024-04-26T04:32:26.542 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 28 v 2 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-04-26T04:32:26.542 INFO:tasks.rados.rados.0.smithi178.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-04-26T04:32:26.542 INFO:tasks.rados.rados.0.smithi178.stdout:28: done (15 left) 2024-04-26T04:32:26.543 INFO:tasks.rados.rados.0.smithi178.stdout:37: write initial oid 37 2024-04-26T04:32:26.543 INFO:tasks.rados.rados.0.smithi178.stdout:37: seq_num 36 ranges {0=606208} 2024-04-26T04:32:26.548 INFO:tasks.rados.rados.0.smithi178.stdout:37: writing smithi17829308-37 from 0 to 606208 tid 1 2024-04-26T04:32:26.548 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.548 INFO:tasks.rados.rados.0.smithi178.stdout:34: finishing write tid 1 to smithi17829308-34 2024-04-26T04:32:26.548 INFO:tasks.rados.rados.0.smithi178.stdout:21: finishing write tid 2 to smithi17829308-21 2024-04-26T04:32:26.550 INFO:tasks.rados.rados.0.smithi178.stdout:22: finishing write tid 3 to smithi17829308-22 2024-04-26T04:32:26.550 INFO:tasks.rados.rados.0.smithi178.stdout:22: oid 22 updating version 0 to 1 2024-04-26T04:32:26.551 INFO:tasks.rados.rados.0.smithi178.stdout:22: oid 22 updating version 1 to 2 2024-04-26T04:32:26.551 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 22 v 2 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-04-26T04:32:26.551 INFO:tasks.rados.rados.0.smithi178.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-04-26T04:32:26.551 INFO:tasks.rados.rados.0.smithi178.stdout:22: done (15 left) 2024-04-26T04:32:26.551 INFO:tasks.rados.rados.0.smithi178.stdout:38: write initial oid 38 2024-04-26T04:32:26.551 INFO:tasks.rados.rados.0.smithi178.stdout:38: seq_num 37 ranges {0=622592} 2024-04-26T04:32:26.554 INFO:tasks.rados.rados.0.smithi178.stdout:38: writing smithi17829308-38 from 0 to 622592 tid 1 2024-04-26T04:32:26.555 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.597 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:26.595+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:26.633 INFO:tasks.rados.rados.0.smithi178.stdout:33: finishing write tid 1 to smithi17829308-33 2024-04-26T04:32:26.636 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:26.636 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:26.646 INFO:tasks.rados.rados.0.smithi178.stdout:29: finishing write tid 1 to smithi17829308-29 2024-04-26T04:32:26.648 INFO:tasks.rados.rados.0.smithi178.stdout:25: finishing write tid 2 to smithi17829308-25 2024-04-26T04:32:26.649 INFO:tasks.rados.rados.0.smithi178.stdout:27: finishing write tid 2 to smithi17829308-27 2024-04-26T04:32:26.652 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T04:32:26.652 INFO:tasks.rados.rados.0.smithi178.stdout:24: finishing write tid 1 to smithi17829308-24 2024-04-26T04:32:26.698 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:26.696+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:26.733 INFO:tasks.rados.rados.0.smithi178.stdout:33: finishing write tid 2 to smithi17829308-33 2024-04-26T04:32:26.733 INFO:tasks.rados.rados.0.smithi178.stdout:24: finishing write tid 2 to smithi17829308-24 2024-04-26T04:32:26.734 INFO:tasks.rados.rados.0.smithi178.stdout:26: finishing write tid 1 to smithi17829308-26 2024-04-26T04:32:26.734 INFO:tasks.rados.rados.0.smithi178.stdout:26: finishing write tid 2 to smithi17829308-26 2024-04-26T04:32:26.741 INFO:tasks.rados.rados.0.smithi178.stdout:34: finishing write tid 2 to smithi17829308-34 2024-04-26T04:32:26.744 INFO:tasks.rados.rados.0.smithi178.stdout:32: finishing write tid 3 to smithi17829308-32 2024-04-26T04:32:26.744 INFO:tasks.rados.rados.0.smithi178.stdout:32: oid 32 updating version 0 to 7 2024-04-26T04:32:26.744 INFO:tasks.rados.rados.0.smithi178.stdout:32: oid 32 updating version 7 to 8 2024-04-26T04:32:26.744 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 32 v 8 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-04-26T04:32:26.744 INFO:tasks.rados.rados.0.smithi178.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-04-26T04:32:26.744 INFO:tasks.rados.rados.0.smithi178.stdout:32: done (15 left) 2024-04-26T04:32:26.745 INFO:tasks.rados.rados.0.smithi178.stdout:39: write initial oid 39 2024-04-26T04:32:26.745 INFO:tasks.rados.rados.0.smithi178.stdout:39: seq_num 38 ranges {0=638976} 2024-04-26T04:32:26.749 INFO:tasks.rados.rados.0.smithi178.stdout:39: writing smithi17829308-39 from 0 to 638976 tid 1 2024-04-26T04:32:26.749 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.749 INFO:tasks.rados.rados.0.smithi178.stdout:29: finishing write tid 2 to smithi17829308-29 2024-04-26T04:32:26.752 INFO:tasks.rados.rados.0.smithi178.stdout:27: finishing write tid 3 to smithi17829308-27 2024-04-26T04:32:26.752 INFO:tasks.rados.rados.0.smithi178.stdout:27: oid 27 updating version 0 to 6 2024-04-26T04:32:26.752 INFO:tasks.rados.rados.0.smithi178.stdout:27: oid 27 version 6 is already newer than 5 2024-04-26T04:32:26.752 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 27 v 6 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-04-26T04:32:26.752 INFO:tasks.rados.rados.0.smithi178.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-04-26T04:32:26.752 INFO:tasks.rados.rados.0.smithi178.stdout:27: done (15 left) 2024-04-26T04:32:26.752 INFO:tasks.rados.rados.0.smithi178.stdout:40: write initial oid 40 2024-04-26T04:32:26.752 INFO:tasks.rados.rados.0.smithi178.stdout:40: seq_num 39 ranges {0=671744} 2024-04-26T04:32:26.757 INFO:tasks.rados.rados.0.smithi178.stdout:40: writing smithi17829308-40 from 0 to 671744 tid 1 2024-04-26T04:32:26.757 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.757 INFO:tasks.rados.rados.0.smithi178.stdout:30: finishing write tid 1 to smithi17829308-30 2024-04-26T04:32:26.759 INFO:tasks.rados.rados.0.smithi178.stdout:36: finishing write tid 1 to smithi17829308-36 2024-04-26T04:32:26.759 INFO:tasks.rados.rados.0.smithi178.stdout:36: finishing write tid 2 to smithi17829308-36 2024-04-26T04:32:26.800 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:26.796+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:26.816 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:26.816 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:26.830 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T04:32:26.850 INFO:tasks.rados.rados.0.smithi178.stdout:21: finishing write tid 3 to smithi17829308-21 2024-04-26T04:32:26.851 INFO:tasks.rados.rados.0.smithi178.stdout:21: oid 21 updating version 0 to 5 2024-04-26T04:32:26.851 INFO:tasks.rados.rados.0.smithi178.stdout:21: oid 21 updating version 5 to 6 2024-04-26T04:32:26.851 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 21 v 6 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-04-26T04:32:26.851 INFO:tasks.rados.rados.0.smithi178.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-04-26T04:32:26.851 INFO:tasks.rados.rados.0.smithi178.stdout:21: done (15 left) 2024-04-26T04:32:26.851 INFO:tasks.rados.rados.0.smithi178.stdout:41: write initial oid 41 2024-04-26T04:32:26.851 INFO:tasks.rados.rados.0.smithi178.stdout:41: seq_num 40 ranges {0=688128} 2024-04-26T04:32:26.855 INFO:tasks.rados.rados.0.smithi178.stdout:41: writing smithi17829308-41 from 0 to 688128 tid 1 2024-04-26T04:32:26.856 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.856 INFO:tasks.rados.rados.0.smithi178.stdout:19: finishing write tid 2 to smithi17829308-19 2024-04-26T04:32:26.856 INFO:tasks.rados.rados.0.smithi178.stdout:30: finishing write tid 2 to smithi17829308-30 2024-04-26T04:32:26.857 INFO:tasks.rados.rados.0.smithi178.stdout:31: finishing write tid 1 to smithi17829308-31 2024-04-26T04:32:26.857 INFO:tasks.rados.rados.0.smithi178.stdout:31: finishing write tid 2 to smithi17829308-31 2024-04-26T04:32:26.859 INFO:tasks.rados.rados.0.smithi178.stdout:34: finishing write tid 3 to smithi17829308-34 2024-04-26T04:32:26.859 INFO:tasks.rados.rados.0.smithi178.stdout:34: oid 34 updating version 0 to 9 2024-04-26T04:32:26.859 INFO:tasks.rados.rados.0.smithi178.stdout:34: oid 34 updating version 9 to 10 2024-04-26T04:32:26.859 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 34 v 10 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-04-26T04:32:26.859 INFO:tasks.rados.rados.0.smithi178.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-04-26T04:32:26.859 INFO:tasks.rados.rados.0.smithi178.stdout:34: done (15 left) 2024-04-26T04:32:26.859 INFO:tasks.rados.rados.0.smithi178.stdout:42: write initial oid 42 2024-04-26T04:32:26.859 INFO:tasks.rados.rados.0.smithi178.stdout:42: seq_num 41 ranges {0=704512} 2024-04-26T04:32:26.871 INFO:tasks.rados.rados.0.smithi178.stdout:42: writing smithi17829308-42 from 0 to 704512 tid 1 2024-04-26T04:32:26.871 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.900 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:26.896+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:26.952 INFO:tasks.rados.rados.0.smithi178.stdout:19: finishing write tid 3 to smithi17829308-19 2024-04-26T04:32:26.952 INFO:tasks.rados.rados.0.smithi178.stdout:19: oid 19 updating version 0 to 5 2024-04-26T04:32:26.953 INFO:tasks.rados.rados.0.smithi178.stdout:19: oid 19 updating version 5 to 6 2024-04-26T04:32:26.953 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 19 v 6 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-04-26T04:32:26.953 INFO:tasks.rados.rados.0.smithi178.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-04-26T04:32:26.953 INFO:tasks.rados.rados.0.smithi178.stdout:19: done (15 left) 2024-04-26T04:32:26.953 INFO:tasks.rados.rados.0.smithi178.stdout:43: write initial oid 43 2024-04-26T04:32:26.953 INFO:tasks.rados.rados.0.smithi178.stdout:43: seq_num 42 ranges {0=720896} 2024-04-26T04:32:26.965 INFO:tasks.rados.rados.0.smithi178.stdout:43: writing smithi17829308-43 from 0 to 720896 tid 1 2024-04-26T04:32:26.965 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:26.965 INFO:tasks.rados.rados.0.smithi178.stdout:25: finishing write tid 3 to smithi17829308-25 2024-04-26T04:32:26.965 INFO:tasks.rados.rados.0.smithi178.stdout:25: oid 25 updating version 0 to 3 2024-04-26T04:32:26.965 INFO:tasks.rados.rados.0.smithi178.stdout:25: oid 25 updating version 3 to 4 2024-04-26T04:32:26.966 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 25 v 4 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-04-26T04:32:26.966 INFO:tasks.rados.rados.0.smithi178.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-04-26T04:32:26.966 INFO:tasks.rados.rados.0.smithi178.stdout:33: finishing write tid 3 to smithi17829308-33 2024-04-26T04:32:26.966 INFO:tasks.rados.rados.0.smithi178.stdout:33: oid 33 updating version 0 to 2 2024-04-26T04:32:26.966 INFO:tasks.rados.rados.0.smithi178.stdout:33: oid 33 version 2 is already newer than 1 2024-04-26T04:32:26.966 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 33 v 2 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-04-26T04:32:26.966 INFO:tasks.rados.rados.0.smithi178.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-04-26T04:32:26.966 INFO:tasks.rados.rados.0.smithi178.stdout:38: finishing write tid 1 to smithi17829308-38 2024-04-26T04:32:26.966 INFO:tasks.rados.rados.0.smithi178.stdout:38: finishing write tid 2 to smithi17829308-38 2024-04-26T04:32:26.966 INFO:tasks.rados.rados.0.smithi178.stdout:25: done (15 left) 2024-04-26T04:32:26.966 INFO:tasks.rados.rados.0.smithi178.stdout:33: done (14 left) 2024-04-26T04:32:26.966 INFO:tasks.rados.rados.0.smithi178.stdout:44: write initial oid 44 2024-04-26T04:32:26.966 INFO:tasks.rados.rados.0.smithi178.stdout:44: seq_num 43 ranges {0=737280} 2024-04-26T04:32:26.967 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:26.967 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:26.977 INFO:tasks.rados.rados.0.smithi178.stdout:44: writing smithi17829308-44 from 0 to 737280 tid 1 2024-04-26T04:32:26.977 INFO:tasks.rados.rados.0.smithi178.stdout:45: write initial oid 45 2024-04-26T04:32:26.977 INFO:tasks.rados.rados.0.smithi178.stdout:45: seq_num 44 ranges {0=753664} 2024-04-26T04:32:26.982 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T04:32:26.990 INFO:tasks.rados.rados.0.smithi178.stdout:45: writing smithi17829308-45 from 0 to 753664 tid 1 2024-04-26T04:32:26.990 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:27.001 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:26.996+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:27.037 INFO:tasks.rados.rados.0.smithi178.stdout:24: finishing write tid 3 to smithi17829308-24 2024-04-26T04:32:27.038 INFO:tasks.rados.rados.0.smithi178.stdout:24: oid 24 updating version 0 to 6 2024-04-26T04:32:27.038 INFO:tasks.rados.rados.0.smithi178.stdout:24: oid 24 version 6 is already newer than 5 2024-04-26T04:32:27.038 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 24 v 6 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-04-26T04:32:27.038 INFO:tasks.rados.rados.0.smithi178.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-04-26T04:32:27.038 INFO:tasks.rados.rados.0.smithi178.stdout:24: done (15 left) 2024-04-26T04:32:27.038 INFO:tasks.rados.rados.0.smithi178.stdout:46: write initial oid 46 2024-04-26T04:32:27.038 INFO:tasks.rados.rados.0.smithi178.stdout:46: seq_num 45 ranges {0=770048} 2024-04-26T04:32:27.044 INFO:tasks.rados.rados.0.smithi178.stdout:46: writing smithi17829308-46 from 0 to 770048 tid 1 2024-04-26T04:32:27.044 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:27.044 INFO:tasks.rados.rados.0.smithi178.stdout:26: finishing write tid 3 to smithi17829308-26 2024-04-26T04:32:27.044 INFO:tasks.rados.rados.0.smithi178.stdout:26: oid 26 updating version 0 to 8 2024-04-26T04:32:27.044 INFO:tasks.rados.rados.0.smithi178.stdout:26: oid 26 version 8 is already newer than 7 2024-04-26T04:32:27.044 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 26 v 8 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-04-26T04:32:27.044 INFO:tasks.rados.rados.0.smithi178.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-04-26T04:32:27.044 INFO:tasks.rados.rados.0.smithi178.stdout:26: done (15 left) 2024-04-26T04:32:27.044 INFO:tasks.rados.rados.0.smithi178.stdout:47: write initial oid 47 2024-04-26T04:32:27.044 INFO:tasks.rados.rados.0.smithi178.stdout:47: seq_num 46 ranges {0=786432} 2024-04-26T04:32:27.050 INFO:tasks.rados.rados.0.smithi178.stdout:47: writing smithi17829308-47 from 0 to 786432 tid 1 2024-04-26T04:32:27.050 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:27.054 INFO:tasks.rados.rados.0.smithi178.stdout:29: finishing write tid 3 to smithi17829308-29 2024-04-26T04:32:27.054 INFO:tasks.rados.rados.0.smithi178.stdout:29: oid 29 updating version 0 to 7 2024-04-26T04:32:27.054 INFO:tasks.rados.rados.0.smithi178.stdout:29: oid 29 updating version 7 to 8 2024-04-26T04:32:27.054 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 29 v 8 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-26T04:32:27.054 INFO:tasks.rados.rados.0.smithi178.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-04-26T04:32:27.054 INFO:tasks.rados.rados.0.smithi178.stdout:29: done (15 left) 2024-04-26T04:32:27.054 INFO:tasks.rados.rados.0.smithi178.stdout:48: write initial oid 48 2024-04-26T04:32:27.054 INFO:tasks.rados.rados.0.smithi178.stdout:48: seq_num 47 ranges {0=425984,425984=376832} 2024-04-26T04:32:27.057 INFO:tasks.rados.rados.0.smithi178.stdout:48: writing smithi17829308-48 from 0 to 425984 tid 1 2024-04-26T04:32:27.060 INFO:tasks.rados.rados.0.smithi178.stdout:48: writing smithi17829308-48 from 425984 to 802816 tid 2 2024-04-26T04:32:27.061 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:27.101 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:27.096+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:27.116 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:27.116 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:27.130 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T04:32:27.148 INFO:tasks.rados.rados.0.smithi178.stdout:36: finishing write tid 3 to smithi17829308-36 2024-04-26T04:32:27.148 INFO:tasks.rados.rados.0.smithi178.stdout:36: oid 36 updating version 0 to 7 2024-04-26T04:32:27.148 INFO:tasks.rados.rados.0.smithi178.stdout:36: oid 36 updating version 7 to 8 2024-04-26T04:32:27.148 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 36 v 8 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-04-26T04:32:27.148 INFO:tasks.rados.rados.0.smithi178.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-04-26T04:32:27.148 INFO:tasks.rados.rados.0.smithi178.stdout:36: done (15 left) 2024-04-26T04:32:27.148 INFO:tasks.rados.rados.0.smithi178.stdout:49: write initial oid 49 2024-04-26T04:32:27.148 INFO:tasks.rados.rados.0.smithi178.stdout:49: seq_num 48 ranges {0=442368,442368=376832} 2024-04-26T04:32:27.150 INFO:tasks.rados.rados.0.smithi178.stdout:49: writing smithi17829308-49 from 0 to 442368 tid 1 2024-04-26T04:32:27.152 INFO:tasks.rados.rados.0.smithi178.stdout:49: writing smithi17829308-49 from 442368 to 819200 tid 2 2024-04-26T04:32:27.153 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:27.166 INFO:tasks.rados.rados.0.smithi178.stdout:39: finishing write tid 1 to smithi17829308-39 2024-04-26T04:32:27.167 INFO:tasks.rados.rados.0.smithi178.stdout:39: finishing write tid 2 to smithi17829308-39 2024-04-26T04:32:27.202 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:27.200+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:27.241 INFO:tasks.rados.rados.0.smithi178.stdout:45: finishing write tid 1 to smithi17829308-45 2024-04-26T04:32:27.243 INFO:tasks.rados.rados.0.smithi178.stdout:44: finishing write tid 1 to smithi17829308-44 2024-04-26T04:32:27.243 INFO:tasks.rados.rados.0.smithi178.stdout:44: finishing write tid 2 to smithi17829308-44 2024-04-26T04:32:27.254 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:27.254 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:27.257 INFO:tasks.rados.rados.0.smithi178.stdout:35: finishing write tid 1 to smithi17829308-35 2024-04-26T04:32:27.258 INFO:tasks.rados.rados.0.smithi178.stdout:35: finishing write tid 2 to smithi17829308-35 2024-04-26T04:32:27.269 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T04:32:27.270 INFO:tasks.rados.rados.0.smithi178.stdout:37: finishing write tid 1 to smithi17829308-37 2024-04-26T04:32:27.270 INFO:tasks.rados.rados.0.smithi178.stdout:30: finishing write tid 3 to smithi17829308-30 2024-04-26T04:32:27.270 INFO:tasks.rados.rados.0.smithi178.stdout:30: oid 30 updating version 0 to 1 2024-04-26T04:32:27.270 INFO:tasks.rados.rados.0.smithi178.stdout:30: oid 30 updating version 1 to 2 2024-04-26T04:32:27.270 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 30 v 2 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-26T04:32:27.270 INFO:tasks.rados.rados.0.smithi178.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-04-26T04:32:27.270 INFO:tasks.rados.rados.0.smithi178.stdout:30: done (15 left) 2024-04-26T04:32:27.270 INFO:tasks.rados.rados.0.smithi178.stdout:50: write initial oid 50 2024-04-26T04:32:27.270 INFO:tasks.rados.rados.0.smithi178.stdout:50: seq_num 49 ranges {0=458752,458752=376832} 2024-04-26T04:32:27.275 INFO:tasks.rados.rados.0.smithi178.stdout:50: writing smithi17829308-50 from 0 to 458752 tid 1 2024-04-26T04:32:27.282 INFO:tasks.rados.rados.0.smithi178.stdout:50: writing smithi17829308-50 from 458752 to 835584 tid 2 2024-04-26T04:32:27.282 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:27.283 INFO:tasks.rados.rados.0.smithi178.stdout:31: finishing write tid 3 to smithi17829308-31 2024-04-26T04:32:27.283 INFO:tasks.rados.rados.0.smithi178.stdout:31: oid 31 updating version 0 to 3 2024-04-26T04:32:27.283 INFO:tasks.rados.rados.0.smithi178.stdout:31: oid 31 updating version 3 to 4 2024-04-26T04:32:27.283 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 31 v 4 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-04-26T04:32:27.283 INFO:tasks.rados.rados.0.smithi178.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-04-26T04:32:27.283 INFO:tasks.rados.rados.0.smithi178.stdout:31: done (15 left) 2024-04-26T04:32:27.283 INFO:tasks.rados.rados.0.smithi178.stdout:51: copy_from oid 2 from oid 11 current snap is 0 2024-04-26T04:32:27.283 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:27.303 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:27.300+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:27.348 INFO:tasks.rados.rados.0.smithi178.stdout:37: finishing write tid 2 to smithi17829308-37 2024-04-26T04:32:27.349 INFO:tasks.rados.rados.0.smithi178.stdout:38: finishing write tid 3 to smithi17829308-38 2024-04-26T04:32:27.350 INFO:tasks.rados.rados.0.smithi178.stdout:38: oid 38 updating version 0 to 9 2024-04-26T04:32:27.350 INFO:tasks.rados.rados.0.smithi178.stdout:38: oid 38 updating version 9 to 10 2024-04-26T04:32:27.350 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 38 v 10 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-04-26T04:32:27.350 INFO:tasks.rados.rados.0.smithi178.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-04-26T04:32:27.350 INFO:tasks.rados.rados.0.smithi178.stdout:38: done (15 left) 2024-04-26T04:32:27.350 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 25 current snap is 0 2024-04-26T04:32:27.350 INFO:tasks.rados.rados.0.smithi178.stdout:52: seq_num 50 ranges {409600=475136,884736=376832} 2024-04-26T04:32:27.356 INFO:tasks.rados.rados.0.smithi178.stdout:52: writing smithi17829308-25 from 409600 to 884736 tid 1 2024-04-26T04:32:27.359 INFO:tasks.rados.rados.0.smithi178.stdout:52: writing smithi17829308-25 from 884736 to 1261568 tid 2 2024-04-26T04:32:27.359 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:27.359 INFO:tasks.rados.rados.0.smithi178.stdout:43: finishing write tid 1 to smithi17829308-43 2024-04-26T04:32:27.360 INFO:tasks.rados.rados.0.smithi178.stdout:40: finishing write tid 1 to smithi17829308-40 2024-04-26T04:32:27.360 INFO:tasks.rados.rados.0.smithi178.stdout:45: finishing write tid 2 to smithi17829308-45 2024-04-26T04:32:27.360 INFO:tasks.rados.rados.0.smithi178.stdout:40: finishing write tid 2 to smithi17829308-40 2024-04-26T04:32:27.381 INFO:tasks.rados.rados.0.smithi178.stdout:41: finishing write tid 1 to smithi17829308-41 2024-04-26T04:32:27.381 INFO:tasks.rados.rados.0.smithi178.stdout:41: finishing write tid 2 to smithi17829308-41 2024-04-26T04:32:27.383 INFO:tasks.rados.rados.0.smithi178.stdout:42: finishing write tid 1 to smithi17829308-42 2024-04-26T04:32:27.383 INFO:tasks.rados.rados.0.smithi178.stdout:42: finishing write tid 2 to smithi17829308-42 2024-04-26T04:32:27.403 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:27.400+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:27.409 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:27.409 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:27.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T04:32:27.450 INFO:tasks.rados.rados.0.smithi178.stdout:46: finishing write tid 1 to smithi17829308-46 2024-04-26T04:32:27.460 INFO:tasks.rados.rados.0.smithi178.stdout:43: finishing write tid 2 to smithi17829308-43 2024-04-26T04:32:27.460 INFO:tasks.rados.rados.0.smithi178.stdout:48: finishing write tid 1 to smithi17829308-48 2024-04-26T04:32:27.461 INFO:tasks.rados.rados.0.smithi178.stdout:47: finishing write tid 1 to smithi17829308-47 2024-04-26T04:32:27.462 INFO:tasks.rados.rados.0.smithi178.stdout:35: finishing write tid 3 to smithi17829308-35 2024-04-26T04:32:27.462 INFO:tasks.rados.rados.0.smithi178.stdout:35: oid 35 updating version 0 to 11 2024-04-26T04:32:27.462 INFO:tasks.rados.rados.0.smithi178.stdout:35: oid 35 updating version 11 to 12 2024-04-26T04:32:27.462 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 35 v 12 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-04-26T04:32:27.463 INFO:tasks.rados.rados.0.smithi178.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-04-26T04:32:27.463 INFO:tasks.rados.rados.0.smithi178.stdout:35: done (15 left) 2024-04-26T04:32:27.463 INFO:tasks.rados.rados.0.smithi178.stdout:53: rmattr oid 30 current snap is 0 2024-04-26T04:32:27.463 INFO:tasks.rados.rados.0.smithi178.stdout:53: done (15 left) 2024-04-26T04:32:27.463 INFO:tasks.rados.rados.0.smithi178.stdout:54: delete oid 3 current snap is 0 2024-04-26T04:32:27.466 INFO:tasks.rados.rados.0.smithi178.stdout:45: finishing write tid 3 to smithi17829308-45 2024-04-26T04:32:27.466 INFO:tasks.rados.rados.0.smithi178.stdout:45: oid 45 updating version 0 to 9 2024-04-26T04:32:27.466 INFO:tasks.rados.rados.0.smithi178.stdout:45: oid 45 updating version 9 to 10 2024-04-26T04:32:27.466 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 45 v 10 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-04-26T04:32:27.466 INFO:tasks.rados.rados.0.smithi178.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-04-26T04:32:27.468 INFO:tasks.rados.rados.0.smithi178.stdout:47: finishing write tid 2 to smithi17829308-47 2024-04-26T04:32:27.486 INFO:tasks.rados.rados.0.smithi178.stdout:50: finishing write tid 1 to smithi17829308-50 2024-04-26T04:32:27.504 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:27.503+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:27.546 INFO:tasks.rados.rados.0.smithi178.stdout:44: finishing write tid 3 to smithi17829308-44 2024-04-26T04:32:27.546 INFO:tasks.rados.rados.0.smithi178.stdout:44: oid 44 updating version 0 to 8 2024-04-26T04:32:27.546 INFO:tasks.rados.rados.0.smithi178.stdout:44: oid 44 version 8 is already newer than 7 2024-04-26T04:32:27.546 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 44 v 8 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-04-26T04:32:27.546 INFO:tasks.rados.rados.0.smithi178.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-04-26T04:32:27.551 INFO:tasks.rados.rados.0.smithi178.stdout:46: finishing write tid 2 to smithi17829308-46 2024-04-26T04:32:27.560 INFO:tasks.rados.rados.0.smithi178.stdout:48: finishing write tid 2 to smithi17829308-48 2024-04-26T04:32:27.560 INFO:tasks.rados.rados.0.smithi178.stdout:48: finishing write tid 3 to smithi17829308-48 2024-04-26T04:32:27.583 INFO:tasks.rados.rados.0.smithi178.stdout:49: finishing write tid 1 to smithi17829308-49 2024-04-26T04:32:27.595 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:27.595 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:27.605 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:27.603+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:27.610 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T04:32:27.655 INFO:tasks.rados.rados.0.smithi178.stdout:46: finishing write tid 3 to smithi17829308-46 2024-04-26T04:32:27.656 INFO:tasks.rados.rados.0.smithi178.stdout:46: oid 46 updating version 0 to 11 2024-04-26T04:32:27.656 INFO:tasks.rados.rados.0.smithi178.stdout:46: oid 46 updating version 11 to 12 2024-04-26T04:32:27.656 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 46 v 12 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-26T04:32:27.656 INFO:tasks.rados.rados.0.smithi178.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-04-26T04:32:27.663 INFO:tasks.rados.rados.0.smithi178.stdout:43: finishing write tid 3 to smithi17829308-43 2024-04-26T04:32:27.663 INFO:tasks.rados.rados.0.smithi178.stdout:43: oid 43 updating version 0 to 5 2024-04-26T04:32:27.664 INFO:tasks.rados.rados.0.smithi178.stdout:43: oid 43 updating version 5 to 6 2024-04-26T04:32:27.664 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 43 v 6 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-04-26T04:32:27.664 INFO:tasks.rados.rados.0.smithi178.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-04-26T04:32:27.665 INFO:tasks.rados.rados.0.smithi178.stdout:40: finishing write tid 3 to smithi17829308-40 2024-04-26T04:32:27.665 INFO:tasks.rados.rados.0.smithi178.stdout:40: oid 40 updating version 0 to 13 2024-04-26T04:32:27.665 INFO:tasks.rados.rados.0.smithi178.stdout:40: oid 40 updating version 13 to 14 2024-04-26T04:32:27.665 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 40 v 14 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-04-26T04:32:27.665 INFO:tasks.rados.rados.0.smithi178.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-04-26T04:32:27.666 INFO:tasks.rados.rados.0.smithi178.stdout:37: finishing write tid 3 to smithi17829308-37 2024-04-26T04:32:27.666 INFO:tasks.rados.rados.0.smithi178.stdout:37: oid 37 updating version 0 to 6 2024-04-26T04:32:27.666 INFO:tasks.rados.rados.0.smithi178.stdout:37: oid 37 version 6 is already newer than 5 2024-04-26T04:32:27.666 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 37 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-26T04:32:27.666 INFO:tasks.rados.rados.0.smithi178.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-04-26T04:32:27.684 INFO:tasks.rados.rados.0.smithi178.stdout:50: finishing write tid 2 to smithi17829308-50 2024-04-26T04:32:27.684 INFO:tasks.rados.rados.0.smithi178.stdout:49: finishing write tid 2 to smithi17829308-49 2024-04-26T04:32:27.684 INFO:tasks.rados.rados.0.smithi178.stdout:49: finishing write tid 3 to smithi17829308-49 2024-04-26T04:32:27.706 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:27.702+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:27.737 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:27.738 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:27.752 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T04:32:27.756 INFO:tasks.rados.rados.0.smithi178.stdout:37: done (15 left) 2024-04-26T04:32:27.756 INFO:tasks.rados.rados.0.smithi178.stdout:40: done (14 left) 2024-04-26T04:32:27.756 INFO:tasks.rados.rados.0.smithi178.stdout:43: done (13 left) 2024-04-26T04:32:27.756 INFO:tasks.rados.rados.0.smithi178.stdout:44: done (12 left) 2024-04-26T04:32:27.756 INFO:tasks.rados.rados.0.smithi178.stdout:45: done (11 left) 2024-04-26T04:32:27.756 INFO:tasks.rados.rados.0.smithi178.stdout:46: done (10 left) 2024-04-26T04:32:27.756 INFO:tasks.rados.rados.0.smithi178.stdout:54: done (9 left) 2024-04-26T04:32:27.756 INFO:tasks.rados.rados.0.smithi178.stdout:55: read oid 20 snap -1 2024-04-26T04:32:27.757 INFO:tasks.rados.rados.0.smithi178.stdout:55: expect (ObjNum 19 snap 0 seq_num 19) 2024-04-26T04:32:27.757 INFO:tasks.rados.rados.0.smithi178.stdout:56: rollback oid 10 current snap is 0 2024-04-26T04:32:27.757 INFO:tasks.rados.rados.0.smithi178.stdout:56: done (10 left) 2024-04-26T04:32:27.757 INFO:tasks.rados.rados.0.smithi178.stdout:57: read oid 7 snap -1 2024-04-26T04:32:27.757 INFO:tasks.rados.rados.0.smithi178.stdout:57: expect (ObjNum 6 snap 0 seq_num 6) 2024-04-26T04:32:27.757 INFO:tasks.rados.rados.0.smithi178.stdout:58: copy_from oid 38 from oid 44 current snap is 0 2024-04-26T04:32:27.757 INFO:tasks.rados.rados.0.smithi178.stdout:59: rollback oid 37 current snap is 0 2024-04-26T04:32:27.757 INFO:tasks.rados.rados.0.smithi178.stdout:59: done (12 left) 2024-04-26T04:32:27.757 INFO:tasks.rados.rados.0.smithi178.stdout:60: copy_from oid 46 from oid 30 current snap is 0 2024-04-26T04:32:27.757 INFO:tasks.rados.rados.0.smithi178.stdout:61: delete oid 29 current snap is 0 2024-04-26T04:32:27.785 INFO:tasks.rados.rados.0.smithi178.stdout:50: finishing write tid 3 to smithi17829308-50 2024-04-26T04:32:27.808 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:27.808+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:27.856 INFO:tasks.rados.rados.0.smithi178.stdout:39: finishing write tid 3 to smithi17829308-39 2024-04-26T04:32:27.856 INFO:tasks.rados.rados.0.smithi178.stdout:39: oid 39 updating version 0 to 1 2024-04-26T04:32:27.857 INFO:tasks.rados.rados.0.smithi178.stdout:39: oid 39 updating version 1 to 2 2024-04-26T04:32:27.857 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 39 v 2 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-26T04:32:27.857 INFO:tasks.rados.rados.0.smithi178.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:32:27.864 INFO:tasks.rados.rados.0.smithi178.stdout:48: finishing write tid 4 to smithi17829308-48 2024-04-26T04:32:27.864 INFO:tasks.rados.rados.0.smithi178.stdout:48: oid 48 updating version 0 to 7 2024-04-26T04:32:27.865 INFO:tasks.rados.rados.0.smithi178.stdout:48: oid 48 updating version 7 to 8 2024-04-26T04:32:27.865 INFO:tasks.rados.rados.0.smithi178.stdout:48: oid 48 updating version 8 to 9 2024-04-26T04:32:27.865 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 48 v 9 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-04-26T04:32:27.865 INFO:tasks.rados.rados.0.smithi178.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-04-26T04:32:27.865 INFO:tasks.rados.rados.0.smithi178.stdout:52: finishing write tid 1 to smithi17829308-25 2024-04-26T04:32:27.866 INFO:tasks.rados.rados.0.smithi178.stdout:52: finishing write tid 2 to smithi17829308-25 2024-04-26T04:32:27.873 INFO:tasks.rados.rados.0.smithi178.stdout:52: finishing write tid 3 to smithi17829308-25 2024-04-26T04:32:27.874 INFO:tasks.rados.rados.0.smithi178.stdout:47: finishing write tid 3 to smithi17829308-47 2024-04-26T04:32:27.874 INFO:tasks.rados.rados.0.smithi178.stdout:47: oid 47 updating version 0 to 8 2024-04-26T04:32:27.874 INFO:tasks.rados.rados.0.smithi178.stdout:47: oid 47 version 8 is already newer than 7 2024-04-26T04:32:27.874 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 47 v 8 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-26T04:32:27.874 INFO:tasks.rados.rados.0.smithi178.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-04-26T04:32:27.880 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:27.880 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:27.896 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T04:32:27.942 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:27.940+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:27.976 INFO:tasks.rados.rados.0.smithi178.stdout:52: finishing write tid 4 to smithi17829308-25 2024-04-26T04:32:27.976 INFO:tasks.rados.rados.0.smithi178.stdout:52: oid 25 updating version 0 to 7 2024-04-26T04:32:27.976 INFO:tasks.rados.rados.0.smithi178.stdout:52: oid 25 version 7 is already newer than 5 2024-04-26T04:32:27.976 INFO:tasks.rados.rados.0.smithi178.stdout:52: oid 25 version 7 is already newer than 6 2024-04-26T04:32:27.976 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 25 v 7 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-04-26T04:32:27.976 INFO:tasks.rados.rados.0.smithi178.stdout:52: left oid 25 (ObjNum 50 snap 0 seq_num 50) 2024-04-26T04:32:27.985 INFO:tasks.rados.rados.0.smithi178.stdout:58: finishing copy_from racing read to smithi17829308-38 2024-04-26T04:32:27.988 INFO:tasks.rados.rados.0.smithi178.stdout:41: finishing write tid 3 to smithi17829308-41 2024-04-26T04:32:27.989 INFO:tasks.rados.rados.0.smithi178.stdout:41: oid 41 updating version 0 to 4 2024-04-26T04:32:27.989 INFO:tasks.rados.rados.0.smithi178.stdout:41: oid 41 version 4 is already newer than 3 2024-04-26T04:32:27.989 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 41 v 4 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-04-26T04:32:27.989 INFO:tasks.rados.rados.0.smithi178.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-04-26T04:32:27.989 INFO:tasks.rados.rados.0.smithi178.stdout:42: finishing write tid 3 to smithi17829308-42 2024-04-26T04:32:27.989 INFO:tasks.rados.rados.0.smithi178.stdout:42: oid 42 updating version 0 to 6 2024-04-26T04:32:27.989 INFO:tasks.rados.rados.0.smithi178.stdout:42: oid 42 version 6 is already newer than 5 2024-04-26T04:32:27.989 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 42 v 6 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-26T04:32:27.989 INFO:tasks.rados.rados.0.smithi178.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-04-26T04:32:28.009 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:28.004+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:28.023 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:28.023 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:28.038 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T04:32:28.060 INFO:tasks.rados.rados.0.smithi178.stdout:39: done (13 left) 2024-04-26T04:32:28.060 INFO:tasks.rados.rados.0.smithi178.stdout:41: done (12 left) 2024-04-26T04:32:28.060 INFO:tasks.rados.rados.0.smithi178.stdout:42: done (11 left) 2024-04-26T04:32:28.060 INFO:tasks.rados.rados.0.smithi178.stdout:47: done (10 left) 2024-04-26T04:32:28.060 INFO:tasks.rados.rados.0.smithi178.stdout:48: done (9 left) 2024-04-26T04:32:28.060 INFO:tasks.rados.rados.0.smithi178.stdout:52: done (8 left) 2024-04-26T04:32:28.060 INFO:tasks.rados.rados.0.smithi178.stdout:61: done (7 left) 2024-04-26T04:32:28.060 INFO:tasks.rados.rados.0.smithi178.stdout:62: read oid 29 snap -1 2024-04-26T04:32:28.060 INFO:tasks.rados.rados.0.smithi178.stdout:62: expect deleted 2024-04-26T04:32:28.060 INFO:tasks.rados.rados.0.smithi178.stdout:63: snap_create 2024-04-26T04:32:28.077 INFO:tasks.rados.rados.0.smithi178.stdout:50: finishing write tid 4 to smithi17829308-50 2024-04-26T04:32:28.077 INFO:tasks.rados.rados.0.smithi178.stdout:50: oid 50 updating version 0 to 5 2024-04-26T04:32:28.077 INFO:tasks.rados.rados.0.smithi178.stdout:50: oid 50 version 5 is already newer than 4 2024-04-26T04:32:28.078 INFO:tasks.rados.rados.0.smithi178.stdout:50: oid 50 version 5 is already newer than 3 2024-04-26T04:32:28.078 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 50 v 5 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-04-26T04:32:28.078 INFO:tasks.rados.rados.0.smithi178.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-04-26T04:32:28.082 INFO:tasks.rados.rados.0.smithi178.stdout:51: finishing copy_from racing read to smithi17829308-2 2024-04-26T04:32:28.086 INFO:tasks.rados.rados.0.smithi178.stdout:58: finishing copy_from to smithi17829308-38 2024-04-26T04:32:28.086 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 38 v 8 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-04-26T04:32:28.088 INFO:tasks.rados.rados.0.smithi178.stdout:49: finishing write tid 4 to smithi17829308-49 2024-04-26T04:32:28.088 INFO:tasks.rados.rados.0.smithi178.stdout:49: oid 49 updating version 0 to 7 2024-04-26T04:32:28.088 INFO:tasks.rados.rados.0.smithi178.stdout:49: oid 49 updating version 7 to 8 2024-04-26T04:32:28.088 INFO:tasks.rados.rados.0.smithi178.stdout:49: oid 49 updating version 8 to 9 2024-04-26T04:32:28.088 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 49 v 9 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-04-26T04:32:28.089 INFO:tasks.rados.rados.0.smithi178.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-04-26T04:32:28.091 INFO:tasks.rados.rados.0.smithi178.stdout:57: expect (ObjNum 6 snap 0 seq_num 6) 2024-04-26T04:32:28.126 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:28.124+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:28.170 INFO:tasks.rados.rados.0.smithi178.stdout:55: expect (ObjNum 19 snap 0 seq_num 19) 2024-04-26T04:32:28.194 INFO:tasks.rados.rados.0.smithi178.stdout:60: finishing copy_from racing read to smithi17829308-46 2024-04-26T04:32:28.204 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T04:32:28.211 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:28.211+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:28.213 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:28.213 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:28.225 INFO:tasks.rados.rados.0.smithi178.stdout:49: done (8 left) 2024-04-26T04:32:28.225 INFO:tasks.rados.rados.0.smithi178.stdout:50: done (7 left) 2024-04-26T04:32:28.225 INFO:tasks.rados.rados.0.smithi178.stdout:55: done (6 left) 2024-04-26T04:32:28.225 INFO:tasks.rados.rados.0.smithi178.stdout:57: done (5 left) 2024-04-26T04:32:28.225 INFO:tasks.rados.rados.0.smithi178.stdout:58: done (4 left) 2024-04-26T04:32:28.225 INFO:tasks.rados.rados.0.smithi178.stdout:62: done (3 left) 2024-04-26T04:32:28.225 INFO:tasks.rados.rados.0.smithi178.stdout:63: done (2 left) 2024-04-26T04:32:28.225 INFO:tasks.rados.rados.0.smithi178.stdout:64: copy_from oid 41 from oid 33 current snap is 1 2024-04-26T04:32:28.226 INFO:tasks.rados.rados.0.smithi178.stdout:65: read oid 4 snap -1 2024-04-26T04:32:28.226 INFO:tasks.rados.rados.0.smithi178.stdout:65: expect (ObjNum 3 snap 0 seq_num 3) 2024-04-26T04:32:28.226 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 29 current snap is 1 2024-04-26T04:32:28.226 INFO:tasks.rados.rados.0.smithi178.stdout:66: seq_num 53 ranges {0=524288,524288=376832} 2024-04-26T04:32:28.231 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T04:32:28.231 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-26T04:32:28.234 INFO:tasks.rados.rados.0.smithi178.stdout:66: writing smithi17829308-29 from 0 to 524288 tid 1 2024-04-26T04:32:28.240 INFO:tasks.rados.rados.0.smithi178.stdout:66: writing smithi17829308-29 from 524288 to 901120 tid 2 2024-04-26T04:32:28.240 INFO:tasks.rados.rados.0.smithi178.stdout:67: rollback oid 44 current snap is 1 2024-04-26T04:32:28.240 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 44 to 0 2024-04-26T04:32:28.240 INFO:tasks.rados.rados.0.smithi178.stdout:68: setattr oid 28 current snap is 1 2024-04-26T04:32:28.243 INFO:tasks.rados.rados.0.smithi178.stdout:69: copy_from oid 23 from oid 16 current snap is 1 2024-04-26T04:32:28.243 INFO:tasks.rados.rados.0.smithi178.stdout:70: read oid 39 snap 0 2024-04-26T04:32:28.243 INFO:tasks.rados.rados.0.smithi178.stdout:70: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:32:28.243 INFO:tasks.rados.rados.0.smithi178.stdout:71: copy_from oid 45 from oid 8 current snap is 1 2024-04-26T04:32:28.244 INFO:tasks.rados.rados.0.smithi178.stdout:72: rollback oid 5 current snap is 1 2024-04-26T04:32:28.244 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 5 to 0 2024-04-26T04:32:28.244 INFO:tasks.rados.rados.0.smithi178.stdout:73: read oid 13 snap 0 2024-04-26T04:32:28.244 INFO:tasks.rados.rados.0.smithi178.stdout:73: expect (ObjNum 12 snap 0 seq_num 12) 2024-04-26T04:32:28.244 INFO:tasks.rados.rados.0.smithi178.stdout:74: rmattr oid 42 current snap is 1 2024-04-26T04:32:28.244 INFO:tasks.rados.rados.0.smithi178.stdout:74: done (12 left) 2024-04-26T04:32:28.244 INFO:tasks.rados.rados.0.smithi178.stdout:75: copy_from oid 1 from oid 37 current snap is 1 2024-04-26T04:32:28.244 INFO:tasks.rados.rados.0.smithi178.stdout:76: read oid 31 snap -1 2024-04-26T04:32:28.244 INFO:tasks.rados.rados.0.smithi178.stdout:76: expect (ObjNum 30 snap 0 seq_num 30) 2024-04-26T04:32:28.244 INFO:tasks.rados.rados.0.smithi178.stdout:77: copy_from oid 3 from oid 15 current snap is 1 2024-04-26T04:32:28.244 INFO:tasks.rados.rados.0.smithi178.stdout:78: rollback oid 21 current snap is 1 2024-04-26T04:32:28.244 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 21 to 0 2024-04-26T04:32:28.245 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:28.312 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:28.311+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:28.344 INFO:tasks.rados.rados.0.smithi178.stdout:51: finishing copy_from to smithi17829308-2 2024-04-26T04:32:28.344 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 2 v 2 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-26T04:32:28.344 INFO:tasks.rados.rados.0.smithi178.stdout:51: done (15 left) 2024-04-26T04:32:28.344 INFO:tasks.rados.rados.0.smithi178.stdout:79: rollback oid 24 current snap is 1 2024-04-26T04:32:28.344 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 24 to 0 2024-04-26T04:32:28.344 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:28.370 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:28.371 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:28.389 INFO:tasks.rados.rados.0.smithi178.stdout:60: finishing copy_from to smithi17829308-46 2024-04-26T04:32:28.389 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 46 v 2 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-26T04:32:28.389 INFO:tasks.rados.rados.0.smithi178.stdout:60: done (15 left) 2024-04-26T04:32:28.389 INFO:tasks.rados.rados.0.smithi178.stdout:80: read oid 14 snap -1 2024-04-26T04:32:28.389 INFO:tasks.rados.rados.0.smithi178.stdout:80: expect (ObjNum 13 snap 0 seq_num 13) 2024-04-26T04:32:28.389 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:28.392 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:28.393 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:28.407 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:28.407 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:28.413 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:28.408+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:28.414 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:28.414 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:28.423 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T04:32:28.438 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:28.438 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:28.454 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 28 v 4 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-04-26T04:32:28.455 INFO:tasks.rados.rados.0.smithi178.stdout:68: done (15 left) 2024-04-26T04:32:28.455 INFO:tasks.rados.rados.0.smithi178.stdout:81: rollback oid 40 current snap is 1 2024-04-26T04:32:28.455 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 40 to 0 2024-04-26T04:32:28.455 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:28.466 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:28.467 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:28.514 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:28.512+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:28.546 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:28.546 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:28.551 INFO:tasks.rados.rados.0.smithi178.stdout:65: expect (ObjNum 3 snap 0 seq_num 3) 2024-04-26T04:32:28.557 INFO:tasks.rados.rados.0.smithi178.stdout:72: finishing rollback tid 0 to smithi17829308-5 2024-04-26T04:32:28.558 INFO:tasks.rados.rados.0.smithi178.stdout:72: finishing rollback tid 1 to smithi17829308-5 2024-04-26T04:32:28.558 INFO:tasks.rados.rados.0.smithi178.stdout:65: done (15 left) 2024-04-26T04:32:28.558 INFO:tasks.rados.rados.0.smithi178.stdout:82: read oid 50 snap 0 2024-04-26T04:32:28.558 INFO:tasks.rados.rados.0.smithi178.stdout:82: expect (ObjNum 49 snap 0 seq_num 49) 2024-04-26T04:32:28.558 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:28.558 INFO:tasks.rados.rados.0.smithi178.stdout:71: finishing copy_from racing read to smithi17829308-45 2024-04-26T04:32:28.559 INFO:tasks.rados.rados.0.smithi178.stdout:79: finishing rollback tid 0 to smithi17829308-24 2024-04-26T04:32:28.559 INFO:tasks.rados.rados.0.smithi178.stdout:79: finishing rollback tid 1 to smithi17829308-24 2024-04-26T04:32:28.564 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T04:32:28.597 INFO:tasks.rados.rados.0.smithi178.stdout:69: finishing copy_from racing read to smithi17829308-23 2024-04-26T04:32:28.615 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:28.615+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:28.630 INFO:tasks.rados.rados.0.smithi178.stdout:67: finishing rollback tid 0 to smithi17829308-44 2024-04-26T04:32:28.658 INFO:tasks.rados.rados.0.smithi178.stdout:72: finishing rollback tid 2 to smithi17829308-5 2024-04-26T04:32:28.658 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 5 v 18 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-26T04:32:28.658 INFO:tasks.rados.rados.0.smithi178.stdout:72: done (15 left) 2024-04-26T04:32:28.659 INFO:tasks.rados.rados.0.smithi178.stdout:83: read oid 35 snap -1 2024-04-26T04:32:28.659 INFO:tasks.rados.rados.0.smithi178.stdout:83: expect (ObjNum 34 snap 0 seq_num 34) 2024-04-26T04:32:28.659 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:28.659 INFO:tasks.rados.rados.0.smithi178.stdout:81: finishing rollback tid 0 to smithi17829308-40 2024-04-26T04:32:28.659 INFO:tasks.rados.rados.0.smithi178.stdout:81: finishing rollback tid 1 to smithi17829308-40 2024-04-26T04:32:28.661 INFO:tasks.rados.rados.0.smithi178.stdout:79: finishing rollback tid 2 to smithi17829308-24 2024-04-26T04:32:28.661 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 24 v 14 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-04-26T04:32:28.661 INFO:tasks.rados.rados.0.smithi178.stdout:79: done (15 left) 2024-04-26T04:32:28.661 INFO:tasks.rados.rados.0.smithi178.stdout:84: read oid 9 snap -1 2024-04-26T04:32:28.661 INFO:tasks.rados.rados.0.smithi178.stdout:84: expect (ObjNum 8 snap 0 seq_num 8) 2024-04-26T04:32:28.662 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:28.690 INFO:tasks.rados.rados.0.smithi178.stdout:67: finishing rollback tid 1 to smithi17829308-44 2024-04-26T04:32:28.694 INFO:tasks.rados.rados.0.smithi178.stdout:80: expect (ObjNum 13 snap 0 seq_num 13) 2024-04-26T04:32:28.712 INFO:tasks.rados.rados.0.smithi178.stdout:80: done (15 left) 2024-04-26T04:32:28.712 INFO:tasks.rados.rados.0.smithi178.stdout:85: read oid 28 snap -1 2024-04-26T04:32:28.712 INFO:tasks.rados.rados.0.smithi178.stdout:85: expect (ObjNum 27 snap 0 seq_num 27) 2024-04-26T04:32:28.713 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:28.716 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:28.714+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:28.724 INFO:tasks.rados.rados.0.smithi178.stdout:73: expect (ObjNum 12 snap 0 seq_num 12) 2024-04-26T04:32:28.736 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:28.736 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:28.740 INFO:tasks.rados.rados.0.smithi178.stdout:77: finishing copy_from racing read to smithi17829308-3 2024-04-26T04:32:28.741 INFO:tasks.rados.rados.0.smithi178.stdout:67: finishing rollback tid 2 to smithi17829308-44 2024-04-26T04:32:28.741 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 44 v 12 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-04-26T04:32:28.741 INFO:tasks.rados.rados.0.smithi178.stdout:64: finishing copy_from racing read to smithi17829308-41 2024-04-26T04:32:28.741 INFO:tasks.rados.rados.0.smithi178.stdout:70: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:32:28.751 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T04:32:28.762 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:28.763 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:28.783 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:28.784 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:28.784 INFO:tasks.rados.rados.0.smithi178.stdout:76: expect (ObjNum 30 snap 0 seq_num 30) 2024-04-26T04:32:28.805 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:28.805 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:28.809 INFO:tasks.rados.rados.0.smithi178.stdout:67: done (15 left) 2024-04-26T04:32:28.809 INFO:tasks.rados.rados.0.smithi178.stdout:70: done (14 left) 2024-04-26T04:32:28.809 INFO:tasks.rados.rados.0.smithi178.stdout:73: done (13 left) 2024-04-26T04:32:28.809 INFO:tasks.rados.rados.0.smithi178.stdout:76: done (12 left) 2024-04-26T04:32:28.809 INFO:tasks.rados.rados.0.smithi178.stdout:86: rollback oid 47 current snap is 1 2024-04-26T04:32:28.809 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 47 to 0 2024-04-26T04:32:28.809 INFO:tasks.rados.rados.0.smithi178.stdout:87: rmattr oid 17 current snap is 1 2024-04-26T04:32:28.810 INFO:tasks.rados.rados.0.smithi178.stdout:87: done (13 left) 2024-04-26T04:32:28.810 INFO:tasks.rados.rados.0.smithi178.stdout:88: delete oid 17 current snap is 1 2024-04-26T04:32:28.817 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:28.812+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:28.822 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:28.822 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:28.824 INFO:tasks.rados.rados.0.smithi178.stdout:66: finishing write tid 1 to smithi17829308-29 2024-04-26T04:32:28.832 INFO:tasks.rados.rados.0.smithi178.stdout:77: finishing copy_from to smithi17829308-3 2024-04-26T04:32:28.832 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 3 v 4 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-04-26T04:32:28.847 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:28.847 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:28.864 INFO:tasks.rados.rados.0.smithi178.stdout:82: expect (ObjNum 49 snap 0 seq_num 49) 2024-04-26T04:32:28.874 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:28.874 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:28.888 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:28.888 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:28.899 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:28.899 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:28.903 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T04:32:28.919 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:28.914+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:28.921 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:28.921 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:28.923 INFO:tasks.rados.rados.0.smithi178.stdout:71: finishing copy_from to smithi17829308-45 2024-04-26T04:32:28.924 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 45 v 2 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-26T04:32:28.924 INFO:tasks.rados.rados.0.smithi178.stdout:81: finishing rollback tid 2 to smithi17829308-40 2024-04-26T04:32:28.924 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 40 v 22 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-04-26T04:32:28.925 INFO:tasks.rados.rados.0.smithi178.stdout:66: finishing write tid 2 to smithi17829308-29 2024-04-26T04:32:28.945 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:28.945 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:28.966 INFO:tasks.rados.rados.0.smithi178.stdout:69: finishing copy_from to smithi17829308-23 2024-04-26T04:32:28.967 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 23 v 6 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-04-26T04:32:28.969 INFO:tasks.rados.rados.0.smithi178.stdout:64: finishing copy_from to smithi17829308-41 2024-04-26T04:32:28.970 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 41 v 2 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-04-26T04:32:28.970 INFO:tasks.rados.rados.0.smithi178.stdout:83: expect (ObjNum 34 snap 0 seq_num 34) 2024-04-26T04:32:29.038 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:29.036+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:29.039 INFO:tasks.rados.rados.0.smithi178.stdout:85: expect (ObjNum 27 snap 0 seq_num 27) 2024-04-26T04:32:29.050 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:29.050 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:32:29.051 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-04-26T04:32:29.051 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-04-26T04:32:29.051 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:29.051 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:29.051 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:29.066 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T04:32:29.079 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:29.080 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:29.122 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:29.118+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:29.162 INFO:tasks.rados.rados.0.smithi178.stdout:75: finishing copy_from racing read to smithi17829308-1 2024-04-26T04:32:29.169 INFO:tasks.rados.rados.0.smithi178.stdout:84: expect (ObjNum 8 snap 0 seq_num 8) 2024-04-26T04:32:29.218 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:29.218 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-04-26T04:32:29.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-04-26T04:32:29.218 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:32:29.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.219 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.8s2 32/30 ECSubWrite(tid=0, reqid=client.4670.0:1, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.219 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.200207+0000", 2024-04-26T04:32:29.219 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.0133224150000002, 2024-04-26T04:32:29.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20152414199999999, 2024-04-26T04:32:29.219 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.219 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.219 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.219 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200207+0000", 2024-04-26T04:32:29.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200207+0000", 2024-04-26T04:32:29.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200207+0000", 2024-04-26T04:32:29.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6000000000000004e-07 2024-04-26T04:32:29.220 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.220 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200301+0000", 2024-04-26T04:32:29.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4094999999999999e-05 2024-04-26T04:32:29.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200346+0000", 2024-04-26T04:32:29.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5096999999999999e-05 2024-04-26T04:32:29.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.221 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.221 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200374+0000", 2024-04-26T04:32:29.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7321000000000002e-05 2024-04-26T04:32:29.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.300611+0000", 2024-04-26T04:32:29.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10023727 2024-04-26T04:32:29.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.300624+0000", 2024-04-26T04:32:29.222 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3145999999999999e-05 2024-04-26T04:32:29.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.401621+0000", 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100996801 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.401731+0000", 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000109852 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.223 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.8s2 32/30 ECSubWrite(tid=1, reqid=client.4670.0:2, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.201963+0000", 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.0115658600000001, 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.337032781, 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.201963+0000", 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.201963+0000", 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.201965+0000", 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9369999999999998e-06 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.202008+0000", 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2861000000000002e-05 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.202055+0000", 2024-04-26T04:32:29.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6623999999999999e-05 2024-04-26T04:32:29.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.202077+0000", 2024-04-26T04:32:29.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2362999999999999e-05 2024-04-26T04:32:29.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.401252+0000", 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.19917501200000001 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.401272+0000", 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9559999999999999e-05 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.538951+0000", 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.137679788 2024-04-26T04:32:29.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.538996+0000", 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4635999999999998e-05 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.8s2 32/30 ECSubWrite(tid=2, reqid=client.4670.0:13, at_version=32'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.204365+0000", 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.0091645189999996, 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.33466354999999998, 2024-04-26T04:32:29.228 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.204365+0000", 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.204365+0000", 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.204366+0000", 2024-04-26T04:32:29.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7799999999999999e-06 2024-04-26T04:32:29.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.204463+0000", 2024-04-26T04:32:29.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7011999999999999e-05 2024-04-26T04:32:29.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.204508+0000", 2024-04-26T04:32:29.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4953000000000001e-05 2024-04-26T04:32:29.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.304601+0000", 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100093188 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.401824+0000", 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.097222188000000001 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.401840+0000", 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5976000000000001e-05 2024-04-26T04:32:29.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.539001+0000", 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.137161229 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.539028+0000", 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7223999999999999e-05 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 32/30 ECSubWrite(tid=0, reqid=client.4670.0:4, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.206154+0000", 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.0073755330000003, 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0021099560000000001, 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.206154+0000", 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.206154+0000", 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.206154+0000", 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.4000000000000001e-07 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.206254+0000", 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9831e-05 2024-04-26T04:32:29.234 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.206309+0000", 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4623999999999997e-05 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.206331+0000", 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2229999999999999e-05 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.206735+0000", 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000404316 2024-04-26T04:32:29.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.206758+0000", 2024-04-26T04:32:29.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2813999999999999e-05 2024-04-26T04:32:29.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.208161+0000", 2024-04-26T04:32:29.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001402534 2024-04-26T04:32:29.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.208263+0000", 2024-04-26T04:32:29.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010296700000000001 2024-04-26T04:32:29.237 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.237 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.237 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.237 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 32/30 ECSubWrite(tid=1, reqid=client.4670.0:5, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.237 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.210411+0000", 2024-04-26T04:32:29.237 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.0031184289999997, 2024-04-26T04:32:29.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10093671999999999, 2024-04-26T04:32:29.237 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.237 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.237 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.210411+0000", 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.210411+0000", 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.210412+0000", 2024-04-26T04:32:29.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5030000000000001e-06 2024-04-26T04:32:29.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.210478+0000", 2024-04-26T04:32:29.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5765000000000005e-05 2024-04-26T04:32:29.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.210508+0000", 2024-04-26T04:32:29.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0375999999999999e-05 2024-04-26T04:32:29.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.210531+0000", 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2844e-05 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.310702+0000", 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100170469 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.310714+0000", 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2023e-05 2024-04-26T04:32:29.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311305+0000", 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00059130399999999996 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311347+0000", 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2435999999999998e-05 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.es6 32/30 ECSubWrite(tid=157, reqid=client.4670.0:31, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.311566+0000", 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.9019631540000002, 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20204892699999999, 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311566+0000", 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311566+0000", 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311567+0000", 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.367e-06 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311906+0000", 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0003388 2024-04-26T04:32:29.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311919+0000", 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3345e-05 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311926+0000", 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6309999999999997e-06 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.412374+0000", 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10044745300000001 2024-04-26T04:32:29.244 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.412396+0000", 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2626999999999999e-05 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.513576+0000", 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101179381 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.513615+0000", 2024-04-26T04:32:29.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9323000000000003e-05 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.8s2 32/30 ECSubWrite(tid=3, reqid=client.4670.0:14, at_version=32'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.342415+0000", 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.8711138169999999, 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.196658685, 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.342415+0000", 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.342415+0000", 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.342415+0000", 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.97e-07 2024-04-26T04:32:29.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.342434+0000", 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8054999999999999e-05 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.342444+0000", 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0506000000000001e-05 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.342454+0000", 2024-04-26T04:32:29.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5119999999999997e-06 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.538170+0000", 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.19571670299999999 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.538177+0000", 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1339999999999998e-06 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.539034+0000", 2024-04-26T04:32:29.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00085620300000000004 2024-04-26T04:32:29.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.539074+0000", 2024-04-26T04:32:29.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0374999999999999e-05 2024-04-26T04:32:29.250 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.250 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.250 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.250 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.es6 32/30 ECSubWrite(tid=158, reqid=client.4670.0:32, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.250 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.413358+0000", 2024-04-26T04:32:29.250 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.8001707009999999, 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10028363999999999, 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413358+0000", 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413358+0000", 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413360+0000", 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.218e-06 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413379+0000", 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9899e-05 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413395+0000", 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5648e-05 2024-04-26T04:32:29.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413411+0000", 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5613000000000001e-05 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413745+0000", 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033377899999999998 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413756+0000", 2024-04-26T04:32:29.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.136e-05 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.513621+0000", 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.099864741000000007 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.513642+0000", 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1382e-05 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.es6 32/30 ECSubWrite(tid=159, reqid=client.4670.0:43, at_version=32'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.413434+0000", 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.800095276, 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100229367, 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413434+0000", 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413434+0000", 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413434+0000", 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1699999999999999e-07 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413526+0000", 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1564000000000006e-05 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413538+0000", 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1756999999999999e-05 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413546+0000", 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.901e-06 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.414149+0000", 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00060251800000000002 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.414171+0000", 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2223000000000001e-05 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.513644+0000", 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.099473275 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.513663+0000", 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8712e-05 2024-04-26T04:32:29.258 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.es6 32/30 ECSubWrite(tid=160, reqid=client.4670.0:44, at_version=32'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.413567+0000", 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.7999617529999998, 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10012215100000001, 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413567+0000", 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413567+0000", 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413568+0000", 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9500000000000003e-07 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413589+0000", 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1291e-05 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413600+0000", 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0411000000000001e-05 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.413607+0000", 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6529999999999994e-06 2024-04-26T04:32:29.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.438208+0000", 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.024601253 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.438218+0000", 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7119999999999995e-06 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.513670+0000", 2024-04-26T04:32:29.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.075452098999999995 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.513689+0000", 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9236999999999999e-05 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.8s2 33/30 ECSubWrite(tid=4, reqid=client.4670.0:34, at_version=33'5, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.449377+0000", 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.7641521760000001, 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.089728112999999998, 2024-04-26T04:32:29.263 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.449377+0000", 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.449377+0000", 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.449379+0000", 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1229999999999998e-06 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.449544+0000", 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016477899999999999 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.449574+0000", 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9734e-05 2024-04-26T04:32:29.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.449590+0000", 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6217000000000001e-05 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.538369+0000", 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.088778882000000003 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.538373+0000", 2024-04-26T04:32:29.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.685e-06 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.539077+0000", 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00070324799999999996 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.539105+0000", 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8445e-05 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.8s2 33/30 ECSubWrite(tid=5, reqid=client.4670.0:35, at_version=33'6, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.451716+0000", 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.7618126529999998, 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.088063345000000001, 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.451716+0000", 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.451716+0000", 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.451718+0000", 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.517e-06 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.451749+0000", 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1063999999999999e-05 2024-04-26T04:32:29.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.451777+0000", 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8062999999999999e-05 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.451794+0000", 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6657999999999999e-05 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.538720+0000", 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.086926474000000004 2024-04-26T04:32:29.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.538733+0000", 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3157e-05 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.539702+0000", 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00096889700000000003 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.539780+0000", 2024-04-26T04:32:29.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7515000000000006e-05 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 33/30 ECSubWrite(tid=2, reqid=client.4670.0:5, at_version=0'0, trim_to=0'0, roll_forward_to=32'2))", 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.612400+0000", 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.6011288299999999, 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001209969, 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.272 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.612400+0000", 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.612400+0000", 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.612401+0000", 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2800000000000002e-07 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.612415+0000", 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4015999999999999e-05 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.612440+0000", 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5228999999999998e-05 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.612455+0000", 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5121e-05 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.612641+0000", 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000185818 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.612654+0000", 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2833999999999999e-05 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.613569+0000", 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00091525199999999997 2024-04-26T04:32:29.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.613610+0000", 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1270999999999998e-05 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpRead(3.8s2 33/30 ECSubRead(tid=6, to_read={3:1c568fb8:::smithi17829308-5:head=0,4096,0}, subchunks={3:1c568fb8:::smithi17829308-5:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.618331+0000", 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.5951984870000002, 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00059872199999999999, 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.276 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.618331+0000", 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.618331+0000", 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.277 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.618332+0000", 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.694e-06 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.618348+0000", 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5696999999999999e-05 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.618381+0000", 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.286e-05 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.618399+0000", 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8314000000000001e-05 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.618533+0000", 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013391499999999999 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.618929+0000", 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00039624199999999999 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.279 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.8s2 33/30 ECSubWrite(tid=7, reqid=client.4670.0:35, at_version=0'0, trim_to=0'0, roll_forward_to=33'6))", 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.719871+0000", 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.4936579600000002, 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010868430000000001, 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.719871+0000", 2024-04-26T04:32:29.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.719871+0000", 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.719873+0000", 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0899999999999999e-06 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.719905+0000", 2024-04-26T04:32:29.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1361e-05 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.719942+0000", 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7713999999999998e-05 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.719968+0000", 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5434000000000001e-05 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.720107+0000", 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000139462 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.720126+0000", 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8660000000000001e-05 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.720928+0000", 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00080221199999999996 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.720958+0000", 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.991e-05 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpRead(3.8s2 33/30 ECSubRead(tid=8, to_read={3:157a1cb7:::smithi17829308-12:head=0,4096,0}, subchunks={3:157a1cb7:::smithi17829308-12:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.720832+0000", 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.492696756, 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10055821500000001, 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.720832+0000", 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.720832+0000", 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.720834+0000", 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3239999999999999e-06 2024-04-26T04:32:29.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.720855+0000", 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1605999999999999e-05 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.720890+0000", 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4498000000000001e-05 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.720926+0000", 2024-04-26T04:32:29.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6263000000000002e-05 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.821104+0000", 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100177665 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.821391+0000", 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028685899999999999 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 33/30 ECSubWrite(tid=4, reqid=client.4670.0:49, at_version=33'3, trim_to=0'0, roll_forward_to=32'2))", 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.814352+0000", 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.3991775610000001, 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0014411980000000001, 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814352+0000", 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814352+0000", 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814353+0000", 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5689999999999999e-06 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814482+0000", 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012847999999999999 2024-04-26T04:32:29.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814511+0000", 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9119999999999999e-05 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814552+0000", 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0831000000000003e-05 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814653+0000", 2024-04-26T04:32:29.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000101216 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814664+0000", 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1121000000000001e-05 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.815738+0000", 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010737399999999999 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.815793+0000", 2024-04-26T04:32:29.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.5121000000000003e-05 2024-04-26T04:32:29.292 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.292 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.292 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.292 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 33/30 ECSubWrite(tid=5, reqid=client.4670.0:50, at_version=33'4, trim_to=0'0, roll_forward_to=32'2))", 2024-04-26T04:32:29.292 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.814600+0000", 2024-04-26T04:32:29.292 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.398929528, 2024-04-26T04:32:29.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001508871, 2024-04-26T04:32:29.292 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.292 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.292 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.292 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814600+0000", 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814600+0000", 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814600+0000", 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2699999999999999e-07 2024-04-26T04:32:29.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814619+0000", 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8461000000000002e-05 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814635+0000", 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6735000000000002e-05 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.814648+0000", 2024-04-26T04:32:29.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2445e-05 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.815206+0000", 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000558206 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.815221+0000", 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4926000000000001e-05 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.816071+0000", 2024-04-26T04:32:29.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000849803 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.816108+0000", 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7567999999999997e-05 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.bs3 35/30 ECSubWrite(tid=57, reqid=client.4670.0:176, at_version=35'11, trim_to=0'0, roll_forward_to=34'9))", 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:28.924416+0000", 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.289113542, 2024-04-26T04:32:29.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001386214, 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.924416+0000", 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.924416+0000", 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.924416+0000", 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3500000000000005e-07 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.924496+0000", 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9993999999999993e-05 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.924514+0000", 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7056000000000001e-05 2024-04-26T04:32:29.298 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.924523+0000", 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5289999999999997e-06 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.924684+0000", 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016124299999999999 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.924694+0000", 2024-04-26T04:32:29.299 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2469999999999997e-06 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.925768+0000", 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001074216 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.925802+0000", 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3994e-05 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.300 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.bs3 35/30 ECSubWrite(tid=59, reqid=client.4670.0:176, at_version=0'0, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:28.969256+0000", 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.244273458, 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00064501700000000001, 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.969256+0000", 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.301 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.969256+0000", 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.969257+0000", 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.486e-06 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.969272+0000", 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4832e-05 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.302 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.969296+0000", 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3603000000000001e-05 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.969306+0000", 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0179e-05 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.969373+0000", 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7545000000000005e-05 2024-04-26T04:32:29.303 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.969384+0000", 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1004e-05 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.969871+0000", 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00048688099999999999 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.969901+0000", 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9487000000000001e-05 2024-04-26T04:32:29.304 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.305 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.305 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.305 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:29.305 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:29.305 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:29.306 INFO:tasks.rados.rados.0.smithi178.stdout:66: finishing write tid 3 to smithi17829308-29 2024-04-26T04:32:29.306 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-04-26T04:32:29.307 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T04:32:29.308 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:29.304+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:29.319 INFO:tasks.rados.rados.0.smithi178.stdout:86: finishing rollback tid 0 to smithi17829308-47 2024-04-26T04:32:29.327 INFO:tasks.rados.rados.0.smithi178.stdout:78: finishing rollback tid 0 to smithi17829308-21 2024-04-26T04:32:29.327 INFO:tasks.rados.rados.0.smithi178.stdout:78: finishing rollback tid 1 to smithi17829308-21 2024-04-26T04:32:29.328 INFO:tasks.rados.rados.0.smithi178.stdout:78: finishing rollback tid 2 to smithi17829308-21 2024-04-26T04:32:29.328 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 21 v 17 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-04-26T04:32:29.364 INFO:tasks.rados.rados.0.smithi178.stdout:75: finishing copy_from to smithi17829308-1 2024-04-26T04:32:29.364 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 1 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-26T04:32:29.407 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:29.407+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:29.419 INFO:tasks.rados.rados.0.smithi178.stdout:86: finishing rollback tid 1 to smithi17829308-47 2024-04-26T04:32:29.419 INFO:tasks.rados.rados.0.smithi178.stdout:86: finishing rollback tid 2 to smithi17829308-47 2024-04-26T04:32:29.420 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 47 v 12 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-26T04:32:29.440 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:29.441 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:29.441 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 0 2024-04-26T04:32:29.441 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:29.455 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T04:32:29.508 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:29.504+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:29.571 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:29.571 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:29.571 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:32:29.571 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:32:29.571 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:29.585 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T04:32:29.631 INFO:tasks.rados.rados.0.smithi178.stdout:66: finishing write tid 4 to smithi17829308-29 2024-04-26T04:32:29.632 INFO:tasks.rados.rados.0.smithi178.stdout:66: oid 29 updating version 0 to 11 2024-04-26T04:32:29.632 INFO:tasks.rados.rados.0.smithi178.stdout:66: oid 29 updating version 11 to 13 2024-04-26T04:32:29.632 INFO:tasks.rados.rados.0.smithi178.stdout:66: oid 29 version 13 is already newer than 12 2024-04-26T04:32:29.632 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 29 v 13 (ObjNum 53 snap 1 seq_num 53) dirty exists 2024-04-26T04:32:29.632 INFO:tasks.rados.rados.0.smithi178.stdout:66: left oid 29 (ObjNum 53 snap 1 seq_num 53) 2024-04-26T04:32:29.633 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:29.630+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:29.710 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:29.707+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:29.712 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:29.712 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:32:29.712 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:32:29.712 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:32:29.712 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.712 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.as2 32/30 ECSubWrite(tid=0, reqid=client.4670.0:40, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.712 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.218256+0000", 2024-04-26T04:32:29.712 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.4921372970000002, 2024-04-26T04:32:29.712 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00197897, 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.218256+0000", 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.218256+0000", 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.713 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.218258+0000", 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6500000000000001e-06 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.218583+0000", 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000325011 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.218703+0000", 2024-04-26T04:32:29.714 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000120383 2024-04-26T04:32:29.715 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.715 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.715 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.715 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.218782+0000", 2024-04-26T04:32:29.715 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8965e-05 2024-04-26T04:32:29.715 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.715 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.715 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.715 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.218918+0000", 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000135566 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.218938+0000", 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0534999999999999e-05 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.220175+0000", 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0012371839999999999 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.716 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.220235+0000", 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.9676000000000003e-05 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.fs1 32/30 ECSubWrite(tid=0, reqid=client.4670.0:7, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.242045+0000", 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.4683483830000004, 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001837904, 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.717 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.242045+0000", 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.242045+0000", 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.718 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.242046+0000", 2024-04-26T04:32:29.719 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0529999999999999e-06 2024-04-26T04:32:29.719 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.719 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.719 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.719 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.242127+0000", 2024-04-26T04:32:29.719 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0674999999999996e-05 2024-04-26T04:32:29.719 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.719 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.719 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.719 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.242161+0000", 2024-04-26T04:32:29.719 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4591999999999997e-05 2024-04-26T04:32:29.719 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.721 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.721 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.721 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.242190+0000", 2024-04-26T04:32:29.721 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8648000000000001e-05 2024-04-26T04:32:29.721 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.721 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.721 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.242363+0000", 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017346000000000001 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.242387+0000", 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3490999999999999e-05 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.243787+0000", 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001400257 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.722 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.243883+0000", 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.5728000000000002e-05 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 35/30 ECSubWrite(tid=20, reqid=client.4670.0:187, at_version=35'12, trim_to=0'0, roll_forward_to=34'8))", 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.448518+0000", 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.2618754860000001, 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0021640940000000001, 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.723 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.448518+0000", 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.448518+0000", 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.724 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.448520+0000", 2024-04-26T04:32:29.725 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.72e-06 2024-04-26T04:32:29.725 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.725 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.725 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.725 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.448589+0000", 2024-04-26T04:32:29.725 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.8973000000000005e-05 2024-04-26T04:32:29.725 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.725 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.726 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.726 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.448635+0000", 2024-04-26T04:32:29.726 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.6270000000000003e-05 2024-04-26T04:32:29.726 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.726 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.726 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.726 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.448660+0000", 2024-04-26T04:32:29.726 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4956e-05 2024-04-26T04:32:29.726 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.726 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.726 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.726 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.448923+0000", 2024-04-26T04:32:29.726 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026298499999999998 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.448947+0000", 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4471999999999999e-05 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.450491+0000", 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0015439119999999999 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.727 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.450682+0000", 2024-04-26T04:32:29.728 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00019080599999999999 2024-04-26T04:32:29.728 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.728 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.728 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.728 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.728 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.728 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.9s1 35/30 ECSubWrite(tid=10, reqid=client.4670.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=35'4))", 2024-04-26T04:32:29.728 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.456992+0000", 2024-04-26T04:32:29.728 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.25340126, 2024-04-26T04:32:29.728 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016572749999999999, 2024-04-26T04:32:29.728 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.728 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.728 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.729 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.729 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.729 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.729 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.456992+0000", 2024-04-26T04:32:29.729 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.729 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.729 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.729 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.729 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.456992+0000", 2024-04-26T04:32:29.729 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.729 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.730 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.730 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.730 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.456994+0000", 2024-04-26T04:32:29.730 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.88e-06 2024-04-26T04:32:29.730 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.730 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.730 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.730 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.457016+0000", 2024-04-26T04:32:29.730 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1861999999999998e-05 2024-04-26T04:32:29.730 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.730 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.730 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.731 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.457044+0000", 2024-04-26T04:32:29.731 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8568000000000002e-05 2024-04-26T04:32:29.731 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.731 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.731 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.731 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.457068+0000", 2024-04-26T04:32:29.731 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3592e-05 2024-04-26T04:32:29.731 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.731 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.457252+0000", 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018442 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.457282+0000", 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9465e-05 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.458576+0000", 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001294536 2024-04-26T04:32:29.732 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.733 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.733 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.733 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.458649+0000", 2024-04-26T04:32:29.733 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2952000000000002e-05 2024-04-26T04:32:29.733 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.733 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.733 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.733 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.733 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.733 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.as2 35/30 ECSubRead(tid=14, to_read={3:57b6ca82:::smithi17829308-14:head=0,57344,0}, subchunks={3:57b6ca82:::smithi17829308-14:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:29.733 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.490464+0000", 2024-04-26T04:32:29.733 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.219929389, 2024-04-26T04:32:29.733 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100735329, 2024-04-26T04:32:29.734 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.734 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.734 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.734 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.734 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.734 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.734 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.490464+0000", 2024-04-26T04:32:29.734 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.734 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.734 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.734 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.734 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.490464+0000", 2024-04-26T04:32:29.734 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.735 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.735 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.735 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.735 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.490466+0000", 2024-04-26T04:32:29.735 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.973e-06 2024-04-26T04:32:29.735 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.735 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.735 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.735 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.490489+0000", 2024-04-26T04:32:29.735 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2849000000000001e-05 2024-04-26T04:32:29.735 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.735 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.736 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.736 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.490531+0000", 2024-04-26T04:32:29.736 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1780999999999999e-05 2024-04-26T04:32:29.736 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.736 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.736 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.736 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.490562+0000", 2024-04-26T04:32:29.736 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1025000000000002e-05 2024-04-26T04:32:29.736 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.736 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.736 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.736 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.590997+0000", 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100435439 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.591199+0000", 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020226200000000001 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 35/30 ECSubWrite(tid=27, reqid=client.4670.0:218, at_version=35'20, trim_to=0'0, roll_forward_to=34'14))", 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.554207+0000", 2024-04-26T04:32:29.737 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.15618647, 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0020803779999999999, 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554207+0000", 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554207+0000", 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.738 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554209+0000", 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1399999999999998e-06 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554263+0000", 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.4023000000000001e-05 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554291+0000", 2024-04-26T04:32:29.739 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7566e-05 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554314+0000", 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3098000000000001e-05 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554492+0000", 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000178324 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554530+0000", 2024-04-26T04:32:29.740 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.7441000000000003e-05 2024-04-26T04:32:29.741 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.741 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.741 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.741 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.556189+0000", 2024-04-26T04:32:29.741 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016595620000000001 2024-04-26T04:32:29.741 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.741 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.741 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.741 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.556287+0000", 2024-04-26T04:32:29.741 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.8224e-05 2024-04-26T04:32:29.741 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.742 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.742 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.742 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.742 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.742 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 35/30 ECSubWrite(tid=28, reqid=client.4670.0:219, at_version=35'21, trim_to=0'0, roll_forward_to=34'14))", 2024-04-26T04:32:29.742 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.554372+0000", 2024-04-26T04:32:29.742 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.156021339, 2024-04-26T04:32:29.742 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10197365, 2024-04-26T04:32:29.742 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.742 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.742 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.742 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.742 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.743 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.743 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554372+0000", 2024-04-26T04:32:29.743 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.743 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.743 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.743 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.743 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554372+0000", 2024-04-26T04:32:29.743 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.743 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.743 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.743 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.743 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554373+0000", 2024-04-26T04:32:29.743 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.063e-06 2024-04-26T04:32:29.744 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.744 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.744 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.744 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554429+0000", 2024-04-26T04:32:29.744 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.6147000000000003e-05 2024-04-26T04:32:29.744 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.744 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.744 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.744 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554447+0000", 2024-04-26T04:32:29.744 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8159e-05 2024-04-26T04:32:29.744 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.744 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.744 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554537+0000", 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.9988000000000006e-05 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.654747+0000", 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10020931199999999 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.654776+0000", 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9085e-05 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.745 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.746 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.746 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.656276+0000", 2024-04-26T04:32:29.746 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001500237 2024-04-26T04:32:29.746 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.746 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.746 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.746 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.656346+0000", 2024-04-26T04:32:29.746 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.9659000000000005e-05 2024-04-26T04:32:29.746 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 35/30 ECSubWrite(tid=29, reqid=client.4670.0:220, at_version=35'22, trim_to=0'0, roll_forward_to=35'17))", 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.657555+0000", 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.0528382220000001, 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001784222, 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.747 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.748 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.657555+0000", 2024-04-26T04:32:29.748 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.748 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.748 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.748 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.748 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.657555+0000", 2024-04-26T04:32:29.748 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.748 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.748 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.748 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.748 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.657557+0000", 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.77e-06 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.657604+0000", 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.6971000000000001e-05 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.657641+0000", 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.7228000000000003e-05 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.749 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.657667+0000", 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5592000000000002e-05 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.657790+0000", 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012364599999999999 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.657808+0000", 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7978999999999998e-05 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.750 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.659225+0000", 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001416692 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.659339+0000", 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000114344 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.751 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.8s3 35/30 ECSubRead(tid=30, to_read={3:148e36d6:::smithi17829308-35:head=0,143360,0}, subchunks={3:148e36d6:::smithi17829308-35:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:29.752 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.660368+0000", 2024-04-26T04:32:29.752 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.050025113, 2024-04-26T04:32:29.752 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10139301100000001, 2024-04-26T04:32:29.752 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.752 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.752 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.752 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.752 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.752 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.752 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.660368+0000", 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.660368+0000", 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.660370+0000", 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.606e-06 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.753 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.660388+0000", 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8586000000000001e-05 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.660454+0000", 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.5843e-05 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.660473+0000", 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8552000000000001e-05 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.754 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.760802+0000", 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100329317 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.761761+0000", 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00095910699999999995 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.8s3 35/30 ECSubRead(tid=31, to_read={3:148e36d6:::smithi17829308-35:head=0,143360,0}, subchunks={3:148e36d6:::smithi17829308-35:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:29.755 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.662160+0000", 2024-04-26T04:32:29.756 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.0482330520000001, 2024-04-26T04:32:29.756 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10000205600000001, 2024-04-26T04:32:29.756 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.756 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.756 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.756 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.662160+0000", 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.662160+0000", 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.662162+0000", 2024-04-26T04:32:29.757 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2500000000000001e-06 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.662175+0000", 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3689000000000001e-05 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.662200+0000", 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4584999999999999e-05 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.758 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.662269+0000", 2024-04-26T04:32:29.759 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.9245999999999994e-05 2024-04-26T04:32:29.759 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.759 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.759 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.759 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.761913+0000", 2024-04-26T04:32:29.759 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.099643732999999998 2024-04-26T04:32:29.759 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.759 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.759 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.759 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.762162+0000", 2024-04-26T04:32:29.759 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024955300000000001 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.8s3 35/30 ECSubRead(tid=32, to_read={3:148e36d6:::smithi17829308-35:head=0,143360,0}, subchunks={3:148e36d6:::smithi17829308-35:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.662679+0000", 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.0477144380000001, 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.099766823000000004, 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.760 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.662679+0000", 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.662679+0000", 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.662681+0000", 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5969999999999999e-06 2024-04-26T04:32:29.761 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.762 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.762 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.762 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.662696+0000", 2024-04-26T04:32:29.762 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5801999999999999e-05 2024-04-26T04:32:29.762 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.762 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.762 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.762 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.662763+0000", 2024-04-26T04:32:29.762 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6552999999999998e-05 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.662792+0000", 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8659000000000001e-05 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.762234+0000", 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.099442317000000002 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.763 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.762446+0000", 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000211895 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.fs1 35/30 ECSubWrite(tid=30, reqid=client.4670.0:207, at_version=35'16, trim_to=0'0, roll_forward_to=34'15))", 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.702102+0000", 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.008291338, 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0019271830000000001, 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.764 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.702102+0000", 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.702102+0000", 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.702104+0000", 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9700000000000002e-06 2024-04-26T04:32:29.765 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.766 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.766 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.766 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.702259+0000", 2024-04-26T04:32:29.766 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000154938 2024-04-26T04:32:29.766 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.766 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.766 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.766 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.702289+0000", 2024-04-26T04:32:29.766 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0408000000000001e-05 2024-04-26T04:32:29.766 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.766 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.766 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.766 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.702305+0000", 2024-04-26T04:32:29.767 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5993000000000001e-05 2024-04-26T04:32:29.767 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.767 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.767 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.767 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.702432+0000", 2024-04-26T04:32:29.767 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012710900000000001 2024-04-26T04:32:29.767 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.767 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.767 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.702463+0000", 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1025999999999997e-05 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.703971+0000", 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00150705 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.704029+0000", 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.8689e-05 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.768 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.9s1 35/30 ECSubRead(tid=13, to_read={3:979b0971:::smithi17829308-28:head=0,114688,0}, subchunks={3:979b0971:::smithi17829308-28:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.817809+0000", 2024-04-26T04:32:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.89258403399999997, 2024-04-26T04:32:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100722432, 2024-04-26T04:32:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.769 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.817809+0000", 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.817809+0000", 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.817811+0000", 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7060000000000001e-06 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.770 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.817827+0000", 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6283e-05 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.817860+0000", 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2838e-05 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.817879+0000", 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8559e-05 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.771 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.772 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.918163+0000", 2024-04-26T04:32:29.772 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10028405 2024-04-26T04:32:29.772 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.772 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.772 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.772 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.918532+0000", 2024-04-26T04:32:29.772 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00036899600000000002 2024-04-26T04:32:29.772 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.772 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.772 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.772 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.773 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.773 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs2 35/30 ECSubWrite(tid=57, reqid=client.4670.0:176, at_version=35'11, trim_to=0'0, roll_forward_to=34'9))", 2024-04-26T04:32:29.773 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.851073+0000", 2024-04-26T04:32:29.773 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.85932086200000002, 2024-04-26T04:32:29.773 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0032647869999999999, 2024-04-26T04:32:29.773 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.773 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.773 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.773 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.773 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.773 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.774 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.851073+0000", 2024-04-26T04:32:29.774 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.774 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.774 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.774 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.774 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.851073+0000", 2024-04-26T04:32:29.774 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.774 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.774 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.774 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.774 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.851075+0000", 2024-04-26T04:32:29.774 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2369999999999999e-06 2024-04-26T04:32:29.774 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.851380+0000", 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00030541599999999998 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.851420+0000", 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.0088e-05 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.851455+0000", 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4508000000000002e-05 2024-04-26T04:32:29.775 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.851646+0000", 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00019163900000000001 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.851671+0000", 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4933e-05 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.854244+0000", 2024-04-26T04:32:29.776 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0025723439999999998 2024-04-26T04:32:29.777 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.777 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.777 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.777 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.854337+0000", 2024-04-26T04:32:29.777 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3622000000000006e-05 2024-04-26T04:32:29.777 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.777 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.777 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.778 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.778 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.778 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 35/30 ECSubWrite(tid=33, reqid=client.4670.0:220, at_version=0'0, trim_to=0'0, roll_forward_to=35'22))", 2024-04-26T04:32:29.778 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.867463+0000", 2024-04-26T04:32:29.778 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.84293053600000001, 2024-04-26T04:32:29.778 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10178548499999999, 2024-04-26T04:32:29.778 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.778 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.778 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.778 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.778 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.778 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.778 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.867463+0000", 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.867463+0000", 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.867465+0000", 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.514e-06 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.779 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.780 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.867496+0000", 2024-04-26T04:32:29.780 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0574000000000002e-05 2024-04-26T04:32:29.780 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.780 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.780 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.780 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.867527+0000", 2024-04-26T04:32:29.780 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1015e-05 2024-04-26T04:32:29.780 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.780 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.780 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.780 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.967671+0000", 2024-04-26T04:32:29.781 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100144222 2024-04-26T04:32:29.781 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.781 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.781 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.781 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.967831+0000", 2024-04-26T04:32:29.781 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016030100000000001 2024-04-26T04:32:29.781 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.781 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.781 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.781 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.967847+0000", 2024-04-26T04:32:29.781 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5415999999999999e-05 2024-04-26T04:32:29.781 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.781 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.969178+0000", 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00133096 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.969248+0000", 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0482999999999996e-05 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.782 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es2 35/30 ECSubWrite(tid=187, reqid=client.4670.0:233, at_version=35'10, trim_to=0'0, roll_forward_to=34'8))", 2024-04-26T04:32:29.783 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.913122+0000", 2024-04-26T04:32:29.783 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.79727181599999997, 2024-04-26T04:32:29.783 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0026156880000000001, 2024-04-26T04:32:29.783 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.783 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.783 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.783 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.783 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.783 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.783 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.913122+0000", 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.913122+0000", 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.913124+0000", 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9230000000000001e-06 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.784 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.785 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.913172+0000", 2024-04-26T04:32:29.785 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.8077000000000001e-05 2024-04-26T04:32:29.785 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.785 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.785 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.785 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.913214+0000", 2024-04-26T04:32:29.785 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.2138000000000002e-05 2024-04-26T04:32:29.785 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.785 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.785 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.785 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.913238+0000", 2024-04-26T04:32:29.785 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4558000000000001e-05 2024-04-26T04:32:29.785 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.786 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.786 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.786 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.913441+0000", 2024-04-26T04:32:29.786 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020315599999999999 2024-04-26T04:32:29.786 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.786 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.786 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.786 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.913467+0000", 2024-04-26T04:32:29.786 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5497999999999999e-05 2024-04-26T04:32:29.786 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.786 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.787 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.787 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.915649+0000", 2024-04-26T04:32:29.787 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.002182167 2024-04-26T04:32:29.787 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.787 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.787 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.787 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.915737+0000", 2024-04-26T04:32:29.787 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.8171000000000007e-05 2024-04-26T04:32:29.787 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.787 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.788 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.788 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.788 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.788 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es2 35/30 ECSubWrite(tid=188, reqid=client.4670.0:234, at_version=35'11, trim_to=0'0, roll_forward_to=34'8))", 2024-04-26T04:32:29.788 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.916096+0000", 2024-04-26T04:32:29.788 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.79429751999999998, 2024-04-26T04:32:29.788 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001800407, 2024-04-26T04:32:29.788 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.788 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.788 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.788 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.788 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.788 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.916096+0000", 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.916096+0000", 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.916098+0000", 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.686e-06 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.789 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.790 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.790 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.916142+0000", 2024-04-26T04:32:29.790 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.4091000000000002e-05 2024-04-26T04:32:29.790 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.790 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.790 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.790 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.916167+0000", 2024-04-26T04:32:29.790 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5249000000000002e-05 2024-04-26T04:32:29.790 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.790 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.790 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.790 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.916184+0000", 2024-04-26T04:32:29.790 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6881999999999999e-05 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.916325+0000", 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014123599999999999 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.916339+0000", 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3569e-05 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.917823+0000", 2024-04-26T04:32:29.791 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001484244 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.917896+0000", 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3449999999999996e-05 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es2 35/30 ECSubWrite(tid=189, reqid=client.4670.0:235, at_version=35'12, trim_to=0'0, roll_forward_to=34'8))", 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.918176+0000", 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.79221710199999995, 2024-04-26T04:32:29.792 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0021368889999999999, 2024-04-26T04:32:29.793 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.793 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.793 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.793 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.793 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.793 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.793 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.918176+0000", 2024-04-26T04:32:29.793 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.793 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.793 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.794 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.794 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.918176+0000", 2024-04-26T04:32:29.794 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.794 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.794 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.794 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.794 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.918178+0000", 2024-04-26T04:32:29.794 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.79e-06 2024-04-26T04:32:29.794 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.794 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.794 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.794 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.918220+0000", 2024-04-26T04:32:29.794 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1794e-05 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.918246+0000", 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5996e-05 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.918267+0000", 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1478999999999999e-05 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.795 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.918462+0000", 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000194279 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.918487+0000", 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5125000000000001e-05 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.920240+0000", 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0017537029999999999 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.796 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.920313+0000", 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2723000000000002e-05 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 35/30 ECSubWrite(tid=34, reqid=client.4670.0:202, at_version=35'24, trim_to=0'0, roll_forward_to=35'22))", 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.161613+0000", 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.548780304, 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.103291546, 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.797 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.798 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.798 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.798 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.161613+0000", 2024-04-26T04:32:29.798 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.798 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.798 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.798 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.798 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.161613+0000", 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.161615+0000", 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.204e-06 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.162484+0000", 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00086872700000000002 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.799 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.162526+0000", 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1791e-05 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.162550+0000", 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4341999999999998e-05 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.262786+0000", 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100236265 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.800 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.262801+0000", 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5012e-05 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.264812+0000", 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.002010333 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.264905+0000", 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.2872000000000001e-05 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.801 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es2 35/30 ECSubWrite(tid=190, reqid=client.4670.0:235, at_version=0'0, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.421930+0000", 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.28846360900000001, 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001813987, 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421930+0000", 2024-04-26T04:32:29.802 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.803 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.803 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.803 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.803 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421930+0000", 2024-04-26T04:32:29.803 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.803 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.803 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.803 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.803 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421933+0000", 2024-04-26T04:32:29.803 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.773e-06 2024-04-26T04:32:29.803 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.803 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.804 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.804 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.422026+0000", 2024-04-26T04:32:29.804 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3675e-05 2024-04-26T04:32:29.804 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.804 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.804 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.804 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.422073+0000", 2024-04-26T04:32:29.804 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.7104999999999997e-05 2024-04-26T04:32:29.804 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.804 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.804 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.804 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.422099+0000", 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6081999999999999e-05 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.422470+0000", 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00037008400000000002 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.422502+0000", 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2474000000000001e-05 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.805 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:29.806 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.423664+0000", 2024-04-26T04:32:29.806 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0011616549999999999 2024-04-26T04:32:29.806 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.806 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.806 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:29.806 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.423744+0000", 2024-04-26T04:32:29.806 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0139e-05 2024-04-26T04:32:29.806 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.806 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.806 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.806 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.806 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.806 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:29.809 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T04:32:29.810 INFO:tasks.rados.rados.0.smithi178.stdout:64: done (13 left) 2024-04-26T04:32:29.810 INFO:tasks.rados.rados.0.smithi178.stdout:66: done (12 left) 2024-04-26T04:32:29.810 INFO:tasks.rados.rados.0.smithi178.stdout:69: done (11 left) 2024-04-26T04:32:29.810 INFO:tasks.rados.rados.0.smithi178.stdout:71: done (10 left) 2024-04-26T04:32:29.810 INFO:tasks.rados.rados.0.smithi178.stdout:75: done (9 left) 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:77: done (8 left) 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:78: done (7 left) 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:81: done (6 left) 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:82: done (5 left) 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:83: done (4 left) 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:84: done (3 left) 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:85: done (2 left) 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:86: done (1 left) 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:88: done (0 left) 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 29 current snap is 1 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:89: seq_num 55 ranges {901120=557056,1458176=376832} 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:89: writing smithi17829308-29 from 901120 to 1458176 tid 1 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:89: writing smithi17829308-29 from 1458176 to 1835008 tid 2 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:90: read oid 13 snap 0 2024-04-26T04:32:29.811 INFO:tasks.rados.rados.0.smithi178.stdout:90: expect (ObjNum 12 snap 0 seq_num 12) 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:91: rmattr oid 23 current snap is 1 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:91: done (2 left) 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:92: read oid 35 snap 0 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:92: expect (ObjNum 34 snap 0 seq_num 34) 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 40 current snap is 1 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:93: seq_num 56 ranges {671744=573440,1245184=376832} 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:93: writing smithi17829308-40 from 671744 to 1245184 tid 1 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:93: writing smithi17829308-40 from 1245184 to 1622016 tid 2 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:94: copy_from oid 17 from oid 50 current snap is 1 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:95: copy_from oid 42 from oid 39 current snap is 1 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 21 current snap is 1 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:96: seq_num 57 ranges {344064=589824,933888=376832} 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:96: writing smithi17829308-21 from 344064 to 933888 tid 1 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:96: writing smithi17829308-21 from 933888 to 1310720 tid 2 2024-04-26T04:32:29.812 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 2 current snap is 1 2024-04-26T04:32:29.813 INFO:tasks.rados.rados.0.smithi178.stdout:97: seq_num 58 ranges {180224=606208,786432=376832} 2024-04-26T04:32:29.813 INFO:tasks.rados.rados.0.smithi178.stdout:97: writing smithi17829308-2 from 180224 to 786432 tid 1 2024-04-26T04:32:29.814 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:29.812+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:29.816 INFO:tasks.rados.rados.0.smithi178.stdout:97: writing smithi17829308-2 from 786432 to 1163264 tid 2 2024-04-26T04:32:29.816 INFO:tasks.rados.rados.0.smithi178.stdout:98: rollback oid 18 current snap is 1 2024-04-26T04:32:29.816 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 18 to 0 2024-04-26T04:32:29.816 INFO:tasks.rados.rados.0.smithi178.stdout:99: delete oid 45 current snap is 1 2024-04-26T04:32:29.915 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:29.915+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:29.986 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:29.986 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:32:29.986 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.986 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:244 3.8s0 3:148e36d6:::smithi17829308-35:2 [read 0~573440,checksum in=4b,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:29.986 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.768310+0000", 2024-04-26T04:32:29.986 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.21742479300000001, 2024-04-26T04:32:29.986 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.21747750299999999, 2024-04-26T04:32:29.986 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 244 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768310+0000", 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.987 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768310+0000", 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768313+0000", 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.779e-06 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768326+0000", 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.258e-05 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.988 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768356+0000", 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0620999999999997e-05 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768371+0000", 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5140000000000001e-05 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768568+0000", 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00019688299999999999 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.989 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768779+0000", 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021095900000000001 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:245 3.8s0 3:148e36d6:::smithi17829308-35:2 [read 0~573440,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.768406+0000", 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.217328982, 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.21750366900000001, 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.990 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.991 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:29.991 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:29.991 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:29.991 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:29.991 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 245 2024-04-26T04:32:29.991 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.991 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.991 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.991 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.991 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768406+0000", 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768406+0000", 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768408+0000", 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4670000000000001e-06 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.992 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768417+0000", 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3230000000000004e-06 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768459+0000", 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1820999999999998e-05 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768483+0000", 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4307999999999998e-05 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.993 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769025+0000", 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000542086 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769073+0000", 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.7707999999999998e-05 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:246 3.8s0 3:148e36d6:::smithi17829308-35:2 [read 0~573440,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.768569+0000", 2024-04-26T04:32:29.994 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.21716608700000001, 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.21749305199999999, 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 246 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768569+0000", 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.995 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768569+0000", 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768570+0000", 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2270000000000001e-06 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768578+0000", 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8520000000000004e-06 2024-04-26T04:32:29.996 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768600+0000", 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1948999999999999e-05 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768618+0000", 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7615999999999999e-05 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769307+0000", 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00068885099999999998 2024-04-26T04:32:29.997 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769389+0000", 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2391000000000004e-05 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=36, attrs_read=0))", 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.770066+0000", 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.215668836, 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.216093269, 2024-04-26T04:32:29.998 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770066+0000", 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770066+0000", 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:29.999 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770067+0000", 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.78e-07 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770288+0000", 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000220583 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770301+0000", 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3219e-05 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.000 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.001 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.001 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870381+0000", 2024-04-26T04:32:30.001 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100080399 2024-04-26T04:32:30.001 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.001 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.001 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.001 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.770296+0000", 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.21543868199999999, 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.215932139, 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770296+0000", 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.002 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770296+0000", 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770298+0000", 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.403e-06 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770560+0000", 2024-04-26T04:32:30.003 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026199900000000001 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770587+0000", 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7126000000000001e-05 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770605+0000", 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8346e-05 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.004 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=38, attrs_read=0))", 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.770642+0000", 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.21509272400000001, 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.21565298399999999, 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770642+0000", 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.005 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770642+0000", 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770644+0000", 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.124e-06 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770906+0000", 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026233199999999999 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.006 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770930+0000", 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4111999999999999e-05 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770944+0000", 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3566e-05 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:247 3.8s0 3.9b533258 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:30.007 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.788189+0000", 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.19754614300000001, 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19817710899999999, 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 247 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.788189+0000", 2024-04-26T04:32:30.008 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.788189+0000", 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.788190+0000", 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0270000000000001e-06 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.789159+0000", 2024-04-26T04:32:30.009 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00096895399999999995 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.789190+0000", 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0839000000000001e-05 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.789205+0000", 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5139e-05 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.010 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:248 3.8s0 3.9b533258 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.793207+0000", 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.19252812699999999, 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19323884899999999, 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 248 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.011 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.012 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.012 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.012 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793207+0000", 2024-04-26T04:32:30.012 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.012 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.012 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.012 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.012 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793207+0000", 2024-04-26T04:32:30.012 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.012 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.013 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.013 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.013 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793209+0000", 2024-04-26T04:32:30.013 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7269999999999999e-06 2024-04-26T04:32:30.013 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.013 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.013 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.013 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793709+0000", 2024-04-26T04:32:30.013 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000500247 2024-04-26T04:32:30.013 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.013 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.013 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.013 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793730+0000", 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0931999999999999e-05 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793741+0000", 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1032999999999999e-05 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:249 3.8s0 3.9b533258 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.793759+0000", 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.19197608899999999, 2024-04-26T04:32:30.014 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19276629300000001, 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 249 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793759+0000", 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.015 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793759+0000", 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793760+0000", 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.043e-06 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793765+0000", 2024-04-26T04:32:30.016 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.51e-06 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793775+0000", 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0586e-05 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893886+0000", 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100111197 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.017 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870075+0000", 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.115659788, 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.116529698, 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870075+0000", 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.018 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870075+0000", 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870075+0000", 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870075+0000", 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.019 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870152+0000", 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.6298999999999996e-05 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870163+0000", 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0892e-05 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.020 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870154+0000", 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.115581374, 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.116516517, 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870154+0000", 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.021 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.022 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870154+0000", 2024-04-26T04:32:30.022 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.022 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.022 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.022 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.022 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870155+0000", 2024-04-26T04:32:30.022 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5489999999999999e-06 2024-04-26T04:32:30.022 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.022 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870320+0000", 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016437400000000001 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870336+0000", 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6013000000000001e-05 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870345+0000", 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3899999999999999e-06 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.023 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=38, attrs_read=0))", 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.115408949, 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.116410559, 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.024 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.025 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870347+0000", 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0727999999999999e-05 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.970445+0000", 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100097987 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.026 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=38, attrs_read=0))", 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870408+0000", 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.115327613, 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.116393996, 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870408+0000", 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.027 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870408+0000", 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870409+0000", 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3090000000000001e-06 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870541+0000", 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000131815 2024-04-26T04:32:30.028 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870551+0000", 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0553e-05 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.970635+0000", 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10008423800000001 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.029 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870458+0000", 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.115277034, 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.11640947, 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870458+0000", 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.030 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870458+0000", 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870459+0000", 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.375e-06 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870589+0000", 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012998600000000001 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.031 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.032 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870610+0000", 2024-04-26T04:32:30.032 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0163999999999999e-05 2024-04-26T04:32:30.032 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.032 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.032 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.032 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870625+0000", 2024-04-26T04:32:30.032 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5339e-05 2024-04-26T04:32:30.032 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.032 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.032 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.032 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=38, attrs_read=0))", 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870645+0000", 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.115089934, 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.116290826, 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870645+0000", 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.033 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.034 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.034 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.034 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870645+0000", 2024-04-26T04:32:30.034 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870646+0000", 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.7800000000000001e-07 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870838+0000", 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00019167500000000001 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870855+0000", 2024-04-26T04:32:30.035 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7730000000000001e-05 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870868+0000", 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2308e-05 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:250 3.8s0 3.9b533258 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.893915+0000", 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.091820634999999998, 2024-04-26T04:32:30.036 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.093092227, 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 250 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893915+0000", 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.037 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.038 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.038 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893915+0000", 2024-04-26T04:32:30.038 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.038 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.038 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.038 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.038 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893917+0000", 2024-04-26T04:32:30.038 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3029999999999998e-06 2024-04-26T04:32:30.038 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.038 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.038 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.038 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893925+0000", 2024-04-26T04:32:30.038 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4370000000000007e-06 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893965+0000", 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9941000000000003e-05 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893976+0000", 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1049999999999999e-05 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.039 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-26T04:32:30.040 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 16 2024-04-26T04:32:30.040 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:30.041 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T04:32:30.042 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:30.036+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:30.072 INFO:tasks.rados.rados.0.smithi178.stdout:92: expect (ObjNum 34 snap 0 seq_num 34) 2024-04-26T04:32:30.115 INFO:tasks.rados.rados.0.smithi178.stdout:90: expect (ObjNum 12 snap 0 seq_num 12) 2024-04-26T04:32:30.130 INFO:tasks.rados.rados.0.smithi178.stdout:95: finishing copy_from racing read to smithi17829308-42 2024-04-26T04:32:30.130 INFO:tasks.rados.rados.0.smithi178.stdout:98: finishing rollback tid 0 to smithi17829308-18 2024-04-26T04:32:30.130 INFO:tasks.rados.rados.0.smithi178.stdout:98: finishing rollback tid 1 to smithi17829308-18 2024-04-26T04:32:30.130 INFO:tasks.rados.rados.0.smithi178.stdout:90: done (9 left) 2024-04-26T04:32:30.130 INFO:tasks.rados.rados.0.smithi178.stdout:92: done (8 left) 2024-04-26T04:32:30.130 INFO:tasks.rados.rados.0.smithi178.stdout:99: done (7 left) 2024-04-26T04:32:30.130 INFO:tasks.rados.rados.0.smithi178.stdout:100: read oid 22 snap -1 2024-04-26T04:32:30.130 INFO:tasks.rados.rados.0.smithi178.stdout:100: expect (ObjNum 21 snap 0 seq_num 21) 2024-04-26T04:32:30.130 INFO:tasks.rados.rados.0.smithi178.stdout:101: rollback oid 31 current snap is 1 2024-04-26T04:32:30.130 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 31 to 0 2024-04-26T04:32:30.131 INFO:tasks.rados.rados.0.smithi178.stdout:102: delete oid 25 current snap is 1 2024-04-26T04:32:30.142 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:30.140+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:30.170 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:30.170 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:30.171 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:32:30.171 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:32:30.171 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:30.185 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T04:32:30.230 INFO:tasks.rados.rados.0.smithi178.stdout:89: finishing write tid 1 to smithi17829308-29 2024-04-26T04:32:30.243 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:30.240+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:30.311 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:30.311 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:32:30.311 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:32:30.311 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:32:30.311 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:1 3.8s0 3:1d1feae2:::smithi17829308-1:head [append 0~32768,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.200828+0000", 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.1093778079999996, 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.311358421, 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 1 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.312 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200828+0000", 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200828+0000", 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200830+0000", 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.57e-06 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.313 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.314 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.314 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200962+0000", 2024-04-26T04:32:30.314 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000132115 2024-04-26T04:32:30.314 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.314 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.314 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.314 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200985+0000", 2024-04-26T04:32:30.314 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3065e-05 2024-04-26T04:32:30.314 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.201007+0000", 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1333000000000001e-05 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.201181+0000", 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017403899999999999 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.201416+0000", 2024-04-26T04:32:30.315 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000234713 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.202587+0000", 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001171727 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.345497+0000", 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.142909339 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:30.316 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.511984+0000", 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.16648707800000001 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.512186+0000", 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000202442 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 32/30 ECSubWriteReply(tid=2, last_complete=32'3, committed=1, applied=1))", 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.212591+0000", 2024-04-26T04:32:30.317 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.0976142739999997, 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.23443240400000001, 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212591+0000", 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212591+0000", 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.318 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.319 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.319 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.319 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212593+0000", 2024-04-26T04:32:30.319 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9240000000000001e-06 2024-04-26T04:32:30.319 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.319 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.319 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.319 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212604+0000", 2024-04-26T04:32:30.319 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1225e-05 2024-04-26T04:32:30.319 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.320 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.320 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.320 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212644+0000", 2024-04-26T04:32:30.320 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9538e-05 2024-04-26T04:32:30.320 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.320 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.320 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.320 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212665+0000", 2024-04-26T04:32:30.320 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1073e-05 2024-04-26T04:32:30.320 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.320 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.320 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.321 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.446985+0000", 2024-04-26T04:32:30.321 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.23432027899999999 2024-04-26T04:32:30.321 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.321 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.321 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.321 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.447024+0000", 2024-04-26T04:32:30.321 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8365000000000003e-05 2024-04-26T04:32:30.321 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.321 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.321 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.321 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.321 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.322 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.0s4 32/30 ECSubWrite(tid=0, reqid=client.4670.0:22, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:30.322 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.302991+0000", 2024-04-26T04:32:30.322 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.0072148199999997, 2024-04-26T04:32:30.322 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.14414888300000001, 2024-04-26T04:32:30.322 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.322 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.322 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.322 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.322 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.322 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.322 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.302991+0000", 2024-04-26T04:32:30.322 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.323 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.323 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.323 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.323 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.302991+0000", 2024-04-26T04:32:30.323 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.323 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.323 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.323 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.323 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.302992+0000", 2024-04-26T04:32:30.323 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3400000000000001e-06 2024-04-26T04:32:30.323 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.323 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.324 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.324 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.303114+0000", 2024-04-26T04:32:30.324 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012154699999999999 2024-04-26T04:32:30.324 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.324 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.324 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.324 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.303143+0000", 2024-04-26T04:32:30.324 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9311000000000001e-05 2024-04-26T04:32:30.324 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.324 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.324 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.324 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.303156+0000", 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3329000000000001e-05 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.346073+0000", 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.042916271999999998 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.346091+0000", 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8241999999999999e-05 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.325 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.447092+0000", 2024-04-26T04:32:30.326 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10100118800000001 2024-04-26T04:32:30.326 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.326 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.326 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.326 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.447140+0000", 2024-04-26T04:32:30.326 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.7654000000000001e-05 2024-04-26T04:32:30.326 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.326 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.326 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.326 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.326 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.326 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 32/30 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-04-26T04:32:30.326 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.305091+0000", 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.0051143299999996, 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.142069055, 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.305091+0000", 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.327 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.305091+0000", 2024-04-26T04:32:30.328 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.328 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.328 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.328 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.328 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.305092+0000", 2024-04-26T04:32:30.328 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7000000000000004e-07 2024-04-26T04:32:30.328 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.328 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.328 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.328 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.305112+0000", 2024-04-26T04:32:30.328 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9698e-05 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.305136+0000", 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4598999999999999e-05 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.305152+0000", 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6118999999999999e-05 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.329 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.447067+0000", 2024-04-26T04:32:30.330 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.14191503799999999 2024-04-26T04:32:30.330 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.330 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.330 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.330 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.447160+0000", 2024-04-26T04:32:30.330 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.2831000000000006e-05 2024-04-26T04:32:30.330 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.330 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 32/30 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.305762+0000", 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.0044436140000004, 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.141471398, 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.331 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.305762+0000", 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.305762+0000", 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.305763+0000", 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.314e-06 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.332 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.305771+0000", 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8930000000000005e-06 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.305797+0000", 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6335000000000001e-05 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.305817+0000", 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9128999999999999e-05 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.334 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.447178+0000", 2024-04-26T04:32:30.334 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.141361177 2024-04-26T04:32:30.334 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.334 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.334 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.334 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.447233+0000", 2024-04-26T04:32:30.334 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.5550000000000002e-05 2024-04-26T04:32:30.334 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.334 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.334 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.334 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.334 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.334 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 32/30 ECSubWriteReply(tid=2, last_complete=32'3, committed=1, applied=1))", 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.306674+0000", 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.0035315909999998, 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.14062165500000001, 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.306674+0000", 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.335 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.306674+0000", 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.306676+0000", 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0499999999999999e-06 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.306697+0000", 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0729000000000001e-05 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.336 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.337 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.306726+0000", 2024-04-26T04:32:30.337 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9268000000000002e-05 2024-04-26T04:32:30.337 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.337 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.337 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.337 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.306741+0000", 2024-04-26T04:32:30.337 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4562999999999999e-05 2024-04-26T04:32:30.337 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.337 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.337 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.447271+0000", 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.14053058299999999 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.447296+0000", 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4462000000000001e-05 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:244 3.8s0 3:148e36d6:::smithi17829308-35:2 [read 0~573440,checksum in=4b,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:30.338 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.768310+0000", 2024-04-26T04:32:30.339 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.54189517600000003, 2024-04-26T04:32:30.339 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.303379758, 2024-04-26T04:32:30.339 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.339 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.339 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:30.339 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:30.339 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:30.339 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:30.339 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 244 2024-04-26T04:32:30.339 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.339 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.341 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.341 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.342 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768310+0000", 2024-04-26T04:32:30.342 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.342 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.342 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.342 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.342 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768310+0000", 2024-04-26T04:32:30.342 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.342 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.342 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.342 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768313+0000", 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.779e-06 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768326+0000", 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.258e-05 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768356+0000", 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0620999999999997e-05 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.343 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768371+0000", 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5140000000000001e-05 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768568+0000", 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00019688299999999999 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768779+0000", 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021095900000000001 2024-04-26T04:32:30.344 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071690+0000", 2024-04-26T04:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30291079599999998 2024-04-26T04:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:245 3.8s0 3:148e36d6:::smithi17829308-35:2 [read 0~573440,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.768406+0000", 2024-04-26T04:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.54179936500000003, 2024-04-26T04:32:30.345 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30361496399999999, 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 245 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768406+0000", 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.346 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768406+0000", 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768408+0000", 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4670000000000001e-06 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768417+0000", 2024-04-26T04:32:30.347 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3230000000000004e-06 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768459+0000", 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1820999999999998e-05 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768483+0000", 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4307999999999998e-05 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.348 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769025+0000", 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000542086 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769073+0000", 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.7707999999999998e-05 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072021+0000", 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.302948251 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.349 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.350 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.350 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:246 3.8s0 3:148e36d6:::smithi17829308-35:2 [read 0~573440,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:30.350 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.768569+0000", 2024-04-26T04:32:30.350 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.54163647000000004, 2024-04-26T04:32:30.350 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30420794400000001, 2024-04-26T04:32:30.350 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.350 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.350 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:30.350 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:30.350 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:30.350 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:30.350 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 246 2024-04-26T04:32:30.350 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768569+0000", 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768569+0000", 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768570+0000", 2024-04-26T04:32:30.351 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2270000000000001e-06 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768578+0000", 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8520000000000004e-06 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768600+0000", 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1948999999999999e-05 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.352 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768618+0000", 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7615999999999999e-05 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769307+0000", 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00068885099999999998 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769389+0000", 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2391000000000004e-05 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.353 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072777+0000", 2024-04-26T04:32:30.354 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30338805800000002 2024-04-26T04:32:30.354 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.354 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.354 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.354 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.354 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.354 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=36, attrs_read=0))", 2024-04-26T04:32:30.354 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.769726+0000", 2024-04-26T04:32:30.354 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.54048000500000004, 2024-04-26T04:32:30.354 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201027758, 2024-04-26T04:32:30.354 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.354 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.354 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769726+0000", 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769726+0000", 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.355 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769726+0000", 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769726+0000", 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769755+0000", 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9039000000000001e-05 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.356 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769768+0000", 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3422e-05 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.970644+0000", 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20087569599999999 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.970753+0000", 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000109601 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.357 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=36, attrs_read=0))", 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.769870+0000", 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.54033586899999997, 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20105009300000001, 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769870+0000", 2024-04-26T04:32:30.358 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769870+0000", 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769871+0000", 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7000000000000003e-07 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.359 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770104+0000", 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00023293700000000001 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770132+0000", 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8368999999999999e-05 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770150+0000", 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8105999999999999e-05 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.360 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.970846+0000", 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20069573099999999 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.970920+0000", 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3980000000000007e-05 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=36, attrs_read=0))", 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.770066+0000", 2024-04-26T04:32:30.361 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.54013921899999995, 2024-04-26T04:32:30.362 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30163910300000002, 2024-04-26T04:32:30.362 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.362 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.362 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.362 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.362 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.362 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.362 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770066+0000", 2024-04-26T04:32:30.362 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.362 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.362 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.362 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.362 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770066+0000", 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770067+0000", 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.78e-07 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770288+0000", 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000220583 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.363 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770301+0000", 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3219e-05 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870381+0000", 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100080399 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071285+0000", 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20090390799999999 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.364 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071705+0000", 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000420116 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.770296+0000", 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.53990906500000002, 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30079836799999998, 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.365 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770296+0000", 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770296+0000", 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770298+0000", 2024-04-26T04:32:30.366 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.403e-06 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770560+0000", 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026199900000000001 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770587+0000", 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7126000000000001e-05 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.367 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770605+0000", 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8346e-05 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071054+0000", 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.300448465 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071095+0000", 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1029000000000003e-05 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.368 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=38, attrs_read=0))", 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.770642+0000", 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.53956310699999999, 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30050239499999998, 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770642+0000", 2024-04-26T04:32:30.369 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770642+0000", 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770644+0000", 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.124e-06 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770906+0000", 2024-04-26T04:32:30.370 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026233199999999999 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770930+0000", 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4111999999999999e-05 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770944+0000", 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3566e-05 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.371 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071117+0000", 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30017346 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071145+0000", 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7801e-05 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870075+0000", 2024-04-26T04:32:30.372 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.44013017100000001, 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20113800600000001, 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870075+0000", 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870075+0000", 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.373 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.374 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.374 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.374 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870075+0000", 2024-04-26T04:32:30.374 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.374 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.374 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.374 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.374 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870075+0000", 2024-04-26T04:32:30.374 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.374 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.374 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.374 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.374 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870152+0000", 2024-04-26T04:32:30.375 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.6298999999999996e-05 2024-04-26T04:32:30.375 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.375 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.375 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.375 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870163+0000", 2024-04-26T04:32:30.375 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0892e-05 2024-04-26T04:32:30.375 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.375 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071183+0000", 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20102083200000001 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071213+0000", 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9983000000000001e-05 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.376 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870154+0000", 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.44005175699999999, 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20111141499999999, 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870154+0000", 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.377 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870154+0000", 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870155+0000", 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5489999999999999e-06 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870320+0000", 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016437400000000001 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.378 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870336+0000", 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6013000000000001e-05 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870345+0000", 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3899999999999999e-06 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071239+0000", 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20089364700000001 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.379 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071265+0000", 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6441999999999999e-05 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=38, attrs_read=0))", 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.43987933200000001, 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20188995300000001, 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.380 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.381 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870347+0000", 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0727999999999999e-05 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.382 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.970445+0000", 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100097987 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072194+0000", 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101748964 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072216+0000", 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2274000000000001e-05 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.383 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=38, attrs_read=0))", 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870408+0000", 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.43979799600000002, 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20239011900000001, 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870408+0000", 2024-04-26T04:32:30.384 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870408+0000", 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870409+0000", 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3090000000000001e-06 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.385 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870541+0000", 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000131815 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870551+0000", 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0553e-05 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.970635+0000", 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10008423800000001 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.386 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072252+0000", 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10161698800000001 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072798+0000", 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00054521599999999995 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-26T04:32:30.387 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870458+0000", 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.43974741699999997, 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201580751, 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870458+0000", 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.388 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870458+0000", 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870459+0000", 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.375e-06 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870589+0000", 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012998600000000001 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.389 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870610+0000", 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0163999999999999e-05 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870625+0000", 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5339e-05 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071721+0000", 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20109591299999999 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.390 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072039+0000", 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000317974 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=38, attrs_read=0))", 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870645+0000", 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.43956031699999998, 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20152561399999999, 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.391 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870645+0000", 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870645+0000", 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870646+0000", 2024-04-26T04:32:30.392 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.7800000000000001e-07 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870838+0000", 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00019167500000000001 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870855+0000", 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7730000000000001e-05 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.393 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870868+0000", 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2308e-05 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072101+0000", 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201233511 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072171+0000", 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.9711999999999999e-05 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.394 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.395 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.395 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:30.396 INFO:tasks.rados.rados.0.smithi178.stdout:98: finishing rollback tid 2 to smithi17829308-18 2024-04-26T04:32:30.396 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 18 v 10 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-04-26T04:32:30.396 INFO:tasks.rados.rados.0.smithi178.stdout:89: finishing write tid 2 to smithi17829308-29 2024-04-26T04:32:30.397 INFO:tasks.rados.rados.0.smithi178.stdout:89: finishing write tid 3 to smithi17829308-29 2024-04-26T04:32:30.397 INFO:tasks.rados.rados.0.smithi178.stdout:93: finishing write tid 1 to smithi17829308-40 2024-04-26T04:32:30.397 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:30.395+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:30.397 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T04:32:30.436 INFO:tasks.rados.rados.0.smithi178.stdout:94: finishing copy_from racing read to smithi17829308-17 2024-04-26T04:32:30.436 INFO:tasks.rados.rados.0.smithi178.stdout:95: finishing copy_from to smithi17829308-42 2024-04-26T04:32:30.436 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 42 v 2 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-26T04:32:30.457 INFO:tasks.rados.rados.0.smithi178.stdout:101: finishing rollback tid 0 to smithi17829308-31 2024-04-26T04:32:30.476 INFO:tasks.rados.rados.0.smithi178.stdout:93: finishing write tid 2 to smithi17829308-40 2024-04-26T04:32:30.476 INFO:tasks.rados.rados.0.smithi178.stdout:93: finishing write tid 3 to smithi17829308-40 2024-04-26T04:32:30.497 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:30.494+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:30.525 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:30.525 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:30.525 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 0 2024-04-26T04:32:30.525 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:30.540 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T04:32:30.559 INFO:tasks.rados.rados.0.smithi178.stdout:101: finishing rollback tid 1 to smithi17829308-31 2024-04-26T04:32:30.560 INFO:tasks.rados.rados.0.smithi178.stdout:96: finishing write tid 1 to smithi17829308-21 2024-04-26T04:32:30.560 INFO:tasks.rados.rados.0.smithi178.stdout:96: finishing write tid 2 to smithi17829308-21 2024-04-26T04:32:30.560 INFO:tasks.rados.rados.0.smithi178.stdout:96: finishing write tid 3 to smithi17829308-21 2024-04-26T04:32:30.561 INFO:tasks.rados.rados.0.smithi178.stdout:93: finishing write tid 4 to smithi17829308-40 2024-04-26T04:32:30.561 INFO:tasks.rados.rados.0.smithi178.stdout:93: oid 40 updating version 0 to 26 2024-04-26T04:32:30.561 INFO:tasks.rados.rados.0.smithi178.stdout:93: oid 40 version 26 is already newer than 25 2024-04-26T04:32:30.561 INFO:tasks.rados.rados.0.smithi178.stdout:93: oid 40 updating version 26 to 27 2024-04-26T04:32:30.561 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 40 v 27 (ObjNum 56 snap 1 seq_num 56) dirty exists 2024-04-26T04:32:30.561 INFO:tasks.rados.rados.0.smithi178.stdout:93: left oid 40 (ObjNum 56 snap 1 seq_num 56) 2024-04-26T04:32:30.562 INFO:tasks.rados.rados.0.smithi178.stdout:97: finishing write tid 1 to smithi17829308-2 2024-04-26T04:32:30.562 INFO:tasks.rados.rados.0.smithi178.stdout:89: finishing write tid 4 to smithi17829308-29 2024-04-26T04:32:30.562 INFO:tasks.rados.rados.0.smithi178.stdout:89: oid 29 updating version 0 to 22 2024-04-26T04:32:30.562 INFO:tasks.rados.rados.0.smithi178.stdout:89: oid 29 version 22 is already newer than 20 2024-04-26T04:32:30.562 INFO:tasks.rados.rados.0.smithi178.stdout:89: oid 29 version 22 is already newer than 21 2024-04-26T04:32:30.562 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 29 v 22 (ObjNum 55 snap 1 seq_num 55) dirty exists 2024-04-26T04:32:30.562 INFO:tasks.rados.rados.0.smithi178.stdout:89: left oid 29 (ObjNum 55 snap 1 seq_num 55) 2024-04-26T04:32:30.598 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:30.596+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:30.660 INFO:tasks.rados.rados.0.smithi178.stdout:101: finishing rollback tid 2 to smithi17829308-31 2024-04-26T04:32:30.660 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 31 v 10 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-04-26T04:32:30.699 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:30.694+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:30.703 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:30.703 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:30.703 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:32:30.703 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:32:30.703 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:30.717 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T04:32:30.761 INFO:tasks.rados.rados.0.smithi178.stdout:97: finishing write tid 2 to smithi17829308-2 2024-04-26T04:32:30.762 INFO:tasks.rados.rados.0.smithi178.stdout:97: finishing write tid 3 to smithi17829308-2 2024-04-26T04:32:30.763 INFO:tasks.rados.rados.0.smithi178.stdout:94: finishing copy_from to smithi17829308-17 2024-04-26T04:32:30.763 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 17 v 5 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-04-26T04:32:30.800 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:30.796+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:30.835 INFO:tasks.rados.rados.0.smithi178.stdout:100: expect (ObjNum 21 snap 0 seq_num 21) 2024-04-26T04:32:30.864 INFO:tasks.rados.rados.0.smithi178.stdout:89: done (9 left) 2024-04-26T04:32:30.864 INFO:tasks.rados.rados.0.smithi178.stdout:93: done (8 left) 2024-04-26T04:32:30.864 INFO:tasks.rados.rados.0.smithi178.stdout:94: done (7 left) 2024-04-26T04:32:30.864 INFO:tasks.rados.rados.0.smithi178.stdout:95: done (6 left) 2024-04-26T04:32:30.864 INFO:tasks.rados.rados.0.smithi178.stdout:98: done (5 left) 2024-04-26T04:32:30.864 INFO:tasks.rados.rados.0.smithi178.stdout:100: done (4 left) 2024-04-26T04:32:30.864 INFO:tasks.rados.rados.0.smithi178.stdout:101: done (3 left) 2024-04-26T04:32:30.864 INFO:tasks.rados.rados.0.smithi178.stdout:102: done (2 left) 2024-04-26T04:32:30.864 INFO:tasks.rados.rados.0.smithi178.stdout:103: read oid 17 snap 0 2024-04-26T04:32:30.865 INFO:tasks.rados.rados.0.smithi178.stdout:103: expect (ObjNum 16 snap 0 seq_num 16) 2024-04-26T04:32:30.865 INFO:tasks.rados.rados.0.smithi178.stdout:104: rollback oid 41 current snap is 1 2024-04-26T04:32:30.865 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 41 to 0 2024-04-26T04:32:30.865 INFO:tasks.rados.rados.0.smithi178.stdout:105: delete oid 8 current snap is 1 2024-04-26T04:32:30.885 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:30.886 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:32:30.886 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:32:30.886 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:32:30.886 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.886 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 32/30 ECSubWrite(tid=0, reqid=client.4670.0:10, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:30.886 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.205329+0000", 2024-04-26T04:32:30.886 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.6788609330000002, 2024-04-26T04:32:30.886 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0059856229999999998, 2024-04-26T04:32:30.886 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.886 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.887 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.887 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.887 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.887 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.887 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.205329+0000", 2024-04-26T04:32:30.887 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.887 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.887 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.887 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.887 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.205329+0000", 2024-04-26T04:32:30.887 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.887 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.888 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.888 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.888 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.205329+0000", 2024-04-26T04:32:30.888 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.3300000000000001e-07 2024-04-26T04:32:30.888 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.888 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.888 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.888 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.205556+0000", 2024-04-26T04:32:30.888 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00022696399999999999 2024-04-26T04:32:30.888 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.888 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.889 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.889 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.205595+0000", 2024-04-26T04:32:30.889 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8754000000000002e-05 2024-04-26T04:32:30.889 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.889 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.889 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.889 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.205620+0000", 2024-04-26T04:32:30.889 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5072e-05 2024-04-26T04:32:30.889 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.889 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.889 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.889 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209547+0000", 2024-04-26T04:32:30.890 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0039271949999999996 2024-04-26T04:32:30.890 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.890 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.890 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.890 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209568+0000", 2024-04-26T04:32:30.890 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0349000000000001e-05 2024-04-26T04:32:30.890 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.890 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.890 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.890 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211255+0000", 2024-04-26T04:32:30.890 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016872759999999999 2024-04-26T04:32:30.891 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.891 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.891 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.891 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211314+0000", 2024-04-26T04:32:30.891 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.9179999999999999e-05 2024-04-26T04:32:30.891 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.891 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.891 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.891 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.891 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.891 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 32/30 ECSubWrite(tid=1, reqid=client.4670.0:11, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:30.892 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.209171+0000", 2024-04-26T04:32:30.892 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.6750190170000003, 2024-04-26T04:32:30.892 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19807992999999999, 2024-04-26T04:32:30.892 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.892 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.892 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.892 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.892 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.892 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.892 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209171+0000", 2024-04-26T04:32:30.892 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.893 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.893 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.893 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.893 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209171+0000", 2024-04-26T04:32:30.893 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.893 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.893 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.893 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.893 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209173+0000", 2024-04-26T04:32:30.893 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1440000000000001e-06 2024-04-26T04:32:30.893 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.893 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209220+0000", 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.7030000000000002e-05 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209250+0000", 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0281999999999999e-05 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209281+0000", 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0895000000000002e-05 2024-04-26T04:32:30.894 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.895 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.895 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.895 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.405953+0000", 2024-04-26T04:32:30.895 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19667172599999999 2024-04-26T04:32:30.895 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.895 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.895 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.895 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.405979+0000", 2024-04-26T04:32:30.895 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6452e-05 2024-04-26T04:32:30.895 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.895 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.895 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.895 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.407083+0000", 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0011043139999999999 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.407250+0000", 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016708700000000001 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s2 32/30 ECSubWrite(tid=1, reqid=client.4670.0:5, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.312729+0000", 2024-04-26T04:32:30.896 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.5714607169999999, 2024-04-26T04:32:30.897 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20126603100000001, 2024-04-26T04:32:30.897 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.897 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.897 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.897 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.897 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.897 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.897 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312729+0000", 2024-04-26T04:32:30.897 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.897 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.897 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.897 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.897 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312729+0000", 2024-04-26T04:32:30.898 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.898 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.898 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.898 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.898 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312731+0000", 2024-04-26T04:32:30.898 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9130000000000001e-06 2024-04-26T04:32:30.898 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.898 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.898 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.898 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312766+0000", 2024-04-26T04:32:30.898 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4768000000000003e-05 2024-04-26T04:32:30.898 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.899 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.899 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.899 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312797+0000", 2024-04-26T04:32:30.899 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1201000000000002e-05 2024-04-26T04:32:30.899 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.899 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.899 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.899 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312814+0000", 2024-04-26T04:32:30.899 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7073000000000001e-05 2024-04-26T04:32:30.899 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.899 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.899 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.900 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.413078+0000", 2024-04-26T04:32:30.900 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10026389400000001 2024-04-26T04:32:30.900 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.900 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.900 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.900 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.413092+0000", 2024-04-26T04:32:30.900 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4188999999999999e-05 2024-04-26T04:32:30.900 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.900 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.900 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.900 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.513794+0000", 2024-04-26T04:32:30.900 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100702298 2024-04-26T04:32:30.900 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.901 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.901 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.901 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.513995+0000", 2024-04-26T04:32:30.901 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000200695 2024-04-26T04:32:30.901 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.901 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.901 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.901 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.901 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.901 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es1 32/30 ECSubWrite(tid=158, reqid=client.4670.0:32, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:30.901 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.316326+0000", 2024-04-26T04:32:30.901 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.5678638039999999, 2024-04-26T04:32:30.901 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102271657, 2024-04-26T04:32:30.902 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.902 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.902 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.902 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.902 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.902 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.902 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316326+0000", 2024-04-26T04:32:30.902 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.902 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.902 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.902 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.902 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316326+0000", 2024-04-26T04:32:30.902 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.903 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.903 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.903 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.903 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316327+0000", 2024-04-26T04:32:30.903 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.454e-06 2024-04-26T04:32:30.903 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.903 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.903 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.903 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316361+0000", 2024-04-26T04:32:30.903 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4060999999999998e-05 2024-04-26T04:32:30.903 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316387+0000", 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5395e-05 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316421+0000", 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4205000000000002e-05 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.416649+0000", 2024-04-26T04:32:30.904 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100228373 2024-04-26T04:32:30.905 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.905 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.905 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.905 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.416673+0000", 2024-04-26T04:32:30.905 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3872000000000001e-05 2024-04-26T04:32:30.905 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.905 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.905 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.905 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.418462+0000", 2024-04-26T04:32:30.905 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0017885710000000001 2024-04-26T04:32:30.905 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.905 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.905 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.906 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.418597+0000", 2024-04-26T04:32:30.906 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013572599999999999 2024-04-26T04:32:30.906 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.906 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.906 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.906 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.906 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.906 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es1 32/30 ECSubWrite(tid=159, reqid=client.4670.0:43, at_version=32'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:30.906 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.318943+0000", 2024-04-26T04:32:30.906 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.5652466990000002, 2024-04-26T04:32:30.906 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100590663, 2024-04-26T04:32:30.906 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.906 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.907 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.907 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.907 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.907 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.907 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.318943+0000", 2024-04-26T04:32:30.907 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.907 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.907 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.907 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.907 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.318943+0000", 2024-04-26T04:32:30.907 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.907 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.318945+0000", 2024-04-26T04:32:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.79e-06 2024-04-26T04:32:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319087+0000", 2024-04-26T04:32:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000142182 2024-04-26T04:32:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.908 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319113+0000", 2024-04-26T04:32:30.909 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5925e-05 2024-04-26T04:32:30.909 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.909 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.909 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.909 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319129+0000", 2024-04-26T04:32:30.909 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5795999999999999e-05 2024-04-26T04:32:30.909 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.909 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.909 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.909 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.417255+0000", 2024-04-26T04:32:30.909 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.098126001000000004 2024-04-26T04:32:30.909 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.909 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.910 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.910 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.417279+0000", 2024-04-26T04:32:30.910 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4729e-05 2024-04-26T04:32:30.910 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.910 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.910 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.910 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.419414+0000", 2024-04-26T04:32:30.910 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0021346199999999998 2024-04-26T04:32:30.910 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.910 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.910 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.910 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.419534+0000", 2024-04-26T04:32:30.910 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00011962 2024-04-26T04:32:30.911 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.911 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.911 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.911 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.911 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.911 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es1 32/30 ECSubWrite(tid=160, reqid=client.4670.0:44, at_version=32'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:30.911 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.321581+0000", 2024-04-26T04:32:30.911 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.5626085380000001, 2024-04-26T04:32:30.911 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.099265378000000001, 2024-04-26T04:32:30.911 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.911 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.911 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.911 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321581+0000", 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321581+0000", 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321583+0000", 2024-04-26T04:32:30.912 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7999999999999999e-06 2024-04-26T04:32:30.913 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.913 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.913 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.913 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321627+0000", 2024-04-26T04:32:30.913 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.4345000000000003e-05 2024-04-26T04:32:30.913 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.913 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.913 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.913 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321666+0000", 2024-04-26T04:32:30.913 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9097000000000002e-05 2024-04-26T04:32:30.913 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.913 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.914 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.914 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321690+0000", 2024-04-26T04:32:30.914 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3849000000000002e-05 2024-04-26T04:32:30.914 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.914 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.914 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.914 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.418676+0000", 2024-04-26T04:32:30.914 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.096986241000000001 2024-04-26T04:32:30.914 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.914 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.914 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.914 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.418698+0000", 2024-04-26T04:32:30.915 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1285999999999999e-05 2024-04-26T04:32:30.915 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.915 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.915 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.915 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.420754+0000", 2024-04-26T04:32:30.915 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.002056422 2024-04-26T04:32:30.915 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.915 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.915 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.915 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.420846+0000", 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.2337999999999996e-05 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 33/30 ECSubWrite(tid=2, reqid=client.4670.0:28, at_version=33'3, trim_to=0'0, roll_forward_to=32'1))", 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.508663+0000", 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.3755260839999996, 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102330728, 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.916 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.917 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.917 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.917 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508663+0000", 2024-04-26T04:32:30.917 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.917 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.917 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.917 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.917 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508663+0000", 2024-04-26T04:32:30.917 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.917 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.917 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.917 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.917 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508664+0000", 2024-04-26T04:32:30.918 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.0999999999999999e-07 2024-04-26T04:32:30.918 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.918 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.918 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.918 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508841+0000", 2024-04-26T04:32:30.918 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017671599999999999 2024-04-26T04:32:30.918 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.918 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.918 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.918 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508866+0000", 2024-04-26T04:32:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4879e-05 2024-04-26T04:32:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508882+0000", 2024-04-26T04:32:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6458999999999999e-05 2024-04-26T04:32:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509022+0000", 2024-04-26T04:32:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014024099999999999 2024-04-26T04:32:30.919 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509036+0000", 2024-04-26T04:32:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.419e-05 2024-04-26T04:32:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.610954+0000", 2024-04-26T04:32:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10191800500000001 2024-04-26T04:32:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.920 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.610994+0000", 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9728000000000003e-05 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 33/30 ECSubWrite(tid=3, reqid=client.4670.0:29, at_version=33'4, trim_to=0'0, roll_forward_to=32'1))", 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.508923+0000", 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.3752670230000001, 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102112141, 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.921 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.922 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.922 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.922 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.922 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508923+0000", 2024-04-26T04:32:30.922 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.922 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.922 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.922 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.922 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508923+0000", 2024-04-26T04:32:30.922 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.922 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.922 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.922 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508923+0000", 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.54e-07 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508943+0000", 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9335e-05 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508976+0000", 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2938000000000002e-05 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.923 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508991+0000", 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5556000000000001e-05 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509908+0000", 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00091681599999999999 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509933+0000", 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5452e-05 2024-04-26T04:32:30.924 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.925 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.925 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.925 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.611002+0000", 2024-04-26T04:32:30.925 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101068475 2024-04-26T04:32:30.925 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.925 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.925 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.925 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.611035+0000", 2024-04-26T04:32:30.925 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2814999999999997e-05 2024-04-26T04:32:30.925 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.925 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.925 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.925 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.926 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.926 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 33/30 ECSubWrite(tid=4, reqid=client.4670.0:46, at_version=33'5, trim_to=0'0, roll_forward_to=32'1))", 2024-04-26T04:32:30.926 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.509007+0000", 2024-04-26T04:32:30.926 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.3751820810000002, 2024-04-26T04:32:30.926 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102066723, 2024-04-26T04:32:30.926 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.926 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.926 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.926 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.926 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.926 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509007+0000", 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509007+0000", 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509008+0000", 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.63e-07 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.927 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509389+0000", 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00038080500000000001 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509419+0000", 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0324999999999998e-05 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509441+0000", 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2181999999999999e-05 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.928 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.929 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.929 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.510486+0000", 2024-04-26T04:32:30.929 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0010447010000000001 2024-04-26T04:32:30.929 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.929 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.929 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.929 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.510509+0000", 2024-04-26T04:32:30.929 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2849000000000001e-05 2024-04-26T04:32:30.929 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.929 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.929 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.929 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.611045+0000", 2024-04-26T04:32:30.929 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10053582699999999 2024-04-26T04:32:30.930 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.930 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.930 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.930 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.611074+0000", 2024-04-26T04:32:30.930 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9570999999999999e-05 2024-04-26T04:32:30.930 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.930 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.930 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.930 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.930 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.930 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 33/30 ECSubWrite(tid=5, reqid=client.4670.0:47, at_version=33'6, trim_to=0'0, roll_forward_to=32'1))", 2024-04-26T04:32:30.930 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.509483+0000", 2024-04-26T04:32:30.930 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.3747067319999999, 2024-04-26T04:32:30.931 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10163465200000001, 2024-04-26T04:32:30.931 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.931 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.931 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.931 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.931 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.931 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.931 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509483+0000", 2024-04-26T04:32:30.931 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.931 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.931 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.931 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.931 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509483+0000", 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509484+0000", 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.336e-06 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509517+0000", 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2864999999999998e-05 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.932 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.933 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509566+0000", 2024-04-26T04:32:30.933 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.9424e-05 2024-04-26T04:32:30.933 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.933 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.933 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.933 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509586+0000", 2024-04-26T04:32:30.933 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9678999999999999e-05 2024-04-26T04:32:30.933 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.933 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.933 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.933 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.511220+0000", 2024-04-26T04:32:30.933 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016340199999999999 2024-04-26T04:32:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.511241+0000", 2024-04-26T04:32:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0894999999999999e-05 2024-04-26T04:32:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.611087+0000", 2024-04-26T04:32:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.099845774999999998 2024-04-26T04:32:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.934 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.611117+0000", 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0657999999999997e-05 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.4s1 33/30 ECSubRead(tid=6, to_read={3:2b198661:::smithi17829308-4:head=0,4096,0}, subchunks={3:2b198661:::smithi17829308-4:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.509624+0000", 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.3745651399999996, 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101322883, 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.935 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509624+0000", 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509624+0000", 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509626+0000", 2024-04-26T04:32:30.936 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.093e-06 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509644+0000", 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8508999999999998e-05 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509677+0000", 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2551000000000001e-05 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.937 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509722+0000", 2024-04-26T04:32:30.938 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.5303999999999999e-05 2024-04-26T04:32:30.938 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.938 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.938 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.938 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.610569+0000", 2024-04-26T04:32:30.938 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10084736800000001 2024-04-26T04:32:30.938 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.938 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.938 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.938 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.610947+0000", 2024-04-26T04:32:30.938 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00037805800000000002 2024-04-26T04:32:30.939 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.939 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.939 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.939 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.939 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.939 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.as3 33/30 ECSubWrite(tid=4, reqid=client.4670.0:52, at_version=33'3, trim_to=0'0, roll_forward_to=32'2))", 2024-04-26T04:32:30.939 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.823740+0000", 2024-04-26T04:32:30.939 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.060449975, 2024-04-26T04:32:30.939 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0029436760000000001, 2024-04-26T04:32:30.939 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.939 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.939 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.939 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.823740+0000", 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.823740+0000", 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.823742+0000", 2024-04-26T04:32:30.940 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.939e-06 2024-04-26T04:32:30.941 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.941 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.941 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.941 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.823945+0000", 2024-04-26T04:32:30.941 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020376199999999999 2024-04-26T04:32:30.941 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.941 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.941 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.941 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.823976+0000", 2024-04-26T04:32:30.941 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0664999999999999e-05 2024-04-26T04:32:30.941 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.941 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.941 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.823992+0000", 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5909e-05 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.824145+0000", 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015307400000000001 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.824178+0000", 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.3117999999999997e-05 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.942 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.826589+0000", 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.002411154 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.826683+0000", 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.4055000000000007e-05 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.943 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.7s2 35/30 ECSubRead(tid=58, to_read={3:e1054bf1:::smithi17829308-37:head=0,151552,0}, subchunks={3:e1054bf1:::smithi17829308-37:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.954839+0000", 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.9293509769999999, 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20146535800000001, 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954839+0000", 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.944 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.945 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954839+0000", 2024-04-26T04:32:30.945 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.945 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.945 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.945 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.945 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954840+0000", 2024-04-26T04:32:30.945 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.826e-06 2024-04-26T04:32:30.945 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.945 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.945 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.945 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954857+0000", 2024-04-26T04:32:30.945 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6333000000000001e-05 2024-04-26T04:32:30.945 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.946 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.946 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.946 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954890+0000", 2024-04-26T04:32:30.946 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.3594999999999999e-05 2024-04-26T04:32:30.946 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.946 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.946 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.946 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954912+0000", 2024-04-26T04:32:30.946 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1325e-05 2024-04-26T04:32:30.946 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.946 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.946 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.946 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.155338+0000", 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20042677 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.156304+0000", 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00096550900000000003 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs6 35/30 ECSubWrite(tid=59, reqid=client.4670.0:176, at_version=0'0, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.055093+0000", 2024-04-26T04:32:30.947 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.82909699, 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201719765, 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055093+0000", 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055093+0000", 2024-04-26T04:32:30.948 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055095+0000", 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.227e-06 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055119+0000", 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4329e-05 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055151+0000", 2024-04-26T04:32:30.949 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1507999999999997e-05 2024-04-26T04:32:30.950 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.950 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.950 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.950 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.155266+0000", 2024-04-26T04:32:30.950 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100114996 2024-04-26T04:32:30.950 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.950 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.950 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.950 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.255601+0000", 2024-04-26T04:32:30.950 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100334886 2024-04-26T04:32:30.950 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.950 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.951 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.951 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.255617+0000", 2024-04-26T04:32:30.951 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6446000000000001e-05 2024-04-26T04:32:30.951 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.951 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.951 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.951 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.256718+0000", 2024-04-26T04:32:30.951 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001101524 2024-04-26T04:32:30.951 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.951 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.951 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.951 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.256812+0000", 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3849000000000002e-05 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es1 35/30 ECSubWrite(tid=190, reqid=client.4670.0:235, at_version=0'0, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.421980+0000", 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.462209825, 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20113165399999999, 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.952 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.953 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.953 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421980+0000", 2024-04-26T04:32:30.953 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.953 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.953 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.953 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.953 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421980+0000", 2024-04-26T04:32:30.953 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.953 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.953 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.953 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.953 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421982+0000", 2024-04-26T04:32:30.953 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2639999999999998e-06 2024-04-26T04:32:30.954 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.954 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.954 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.954 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.422038+0000", 2024-04-26T04:32:30.954 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.6273000000000001e-05 2024-04-26T04:32:30.954 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.954 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.954 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.954 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.422071+0000", 2024-04-26T04:32:30.954 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2437999999999997e-05 2024-04-26T04:32:30.954 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.954 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.957 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.957 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.522157+0000", 2024-04-26T04:32:30.957 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100086417 2024-04-26T04:32:30.957 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.957 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.957 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.622323+0000", 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100166121 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.622332+0000", 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.1309999999999995e-06 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.623023+0000", 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000690491 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.958 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.959 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.959 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.623111+0000", 2024-04-26T04:32:30.959 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.8518999999999997e-05 2024-04-26T04:32:30.959 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.959 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.959 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.959 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.959 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.959 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.as3 36/30 ECSubWrite(tid=15, reqid=client.4670.0:263, at_version=36'8, trim_to=0'0, roll_forward_to=34'6))", 2024-04-26T04:32:30.959 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.820102+0000", 2024-04-26T04:32:30.959 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.0640876399999999, 2024-04-26T04:32:30.959 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10193362, 2024-04-26T04:32:30.959 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.820102+0000", 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.820102+0000", 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.960 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.820104+0000", 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0229999999999999e-06 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.820143+0000", 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8794000000000002e-05 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.820175+0000", 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2045999999999998e-05 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.961 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.962 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.962 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.920287+0000", 2024-04-26T04:32:30.962 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10011241899999999 2024-04-26T04:32:30.962 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.962 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.962 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.962 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.920421+0000", 2024-04-26T04:32:30.962 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013344 2024-04-26T04:32:30.962 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.962 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.962 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.962 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.920439+0000", 2024-04-26T04:32:30.963 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8168999999999998e-05 2024-04-26T04:32:30.963 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.963 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.963 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.963 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.921977+0000", 2024-04-26T04:32:30.963 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0015382519999999999 2024-04-26T04:32:30.963 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.963 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.963 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.963 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.922036+0000", 2024-04-26T04:32:30.963 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.8477000000000003e-05 2024-04-26T04:32:30.963 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.963 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s2 36/30 ECSubWrite(tid=64, reqid=client.4670.0:237, at_version=36'20, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.943082+0000", 2024-04-26T04:32:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.941107095, 2024-04-26T04:32:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102547737, 2024-04-26T04:32:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.964 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.943082+0000", 2024-04-26T04:32:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.943082+0000", 2024-04-26T04:32:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.965 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.943085+0000", 2024-04-26T04:32:30.966 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1330000000000002e-06 2024-04-26T04:32:30.966 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.966 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.966 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.966 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.943380+0000", 2024-04-26T04:32:30.966 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00029546100000000001 2024-04-26T04:32:30.966 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.966 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.966 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.966 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.943426+0000", 2024-04-26T04:32:30.966 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.6224000000000003e-05 2024-04-26T04:32:30.966 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.966 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.043544+0000", 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10011753399999999 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.043755+0000", 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021160100000000001 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.043780+0000", 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4454999999999999e-05 2024-04-26T04:32:30.967 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.968 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.968 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.968 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.045600+0000", 2024-04-26T04:32:30.968 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001819878 2024-04-26T04:32:30.968 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.968 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.968 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.968 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.045630+0000", 2024-04-26T04:32:30.968 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0451e-05 2024-04-26T04:32:30.968 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.968 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.969 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.969 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.969 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.969 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs6 36/30 ECSubWrite(tid=71, reqid=client.4670.0:253, at_version=36'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T04:32:30.969 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.106400+0000", 2024-04-26T04:32:30.969 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.77778952899999998, 2024-04-26T04:32:30.969 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101669039, 2024-04-26T04:32:30.969 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.969 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.969 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.969 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.970 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.970 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.970 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.106400+0000", 2024-04-26T04:32:30.970 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.970 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.970 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.970 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.970 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.106400+0000", 2024-04-26T04:32:30.970 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.970 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.970 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.970 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.106401+0000", 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.9399999999999993e-07 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.106594+0000", 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00019320399999999999 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.106608+0000", 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4157e-05 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.971 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.972 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.972 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.106617+0000", 2024-04-26T04:32:30.972 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.9420000000000001e-06 2024-04-26T04:32:30.972 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.972 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.972 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.972 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.206757+0000", 2024-04-26T04:32:30.972 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100139756 2024-04-26T04:32:30.972 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.972 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.972 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.972 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.206766+0000", 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.1050000000000001e-06 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.208029+0000", 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0012624450000000001 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.208069+0000", 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.0435999999999997e-05 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.973 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.974 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.974 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.974 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s2 36/30 ECSubWrite(tid=74, reqid=client.4670.0:260, at_version=36'28, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:30.974 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.166528+0000", 2024-04-26T04:32:30.974 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.71766116800000002, 2024-04-26T04:32:30.974 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10149751999999999, 2024-04-26T04:32:30.974 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.974 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.974 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.974 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.974 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.974 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.166528+0000", 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.166528+0000", 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.166530+0000", 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.494e-06 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.975 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.166627+0000", 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7450999999999999e-05 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.166655+0000", 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7563e-05 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.166670+0000", 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4959e-05 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.976 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.266863+0000", 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100192948 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.266874+0000", 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1065e-05 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.267924+0000", 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0010500749999999999 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.977 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.978 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.268026+0000", 2024-04-26T04:32:30.978 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000101965 2024-04-26T04:32:30.978 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.978 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.978 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.978 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.978 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.978 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s2 36/30 ECSubWrite(tid=79, reqid=client.4670.0:251, at_version=36'30, trim_to=0'0, roll_forward_to=36'22))", 2024-04-26T04:32:30.978 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.378048+0000", 2024-04-26T04:32:30.978 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.50614201299999995, 2024-04-26T04:32:30.978 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10203366699999999, 2024-04-26T04:32:30.978 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:30.978 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.378048+0000", 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.378048+0000", 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.979 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.378049+0000", 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.125e-06 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.378282+0000", 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00023344899999999999 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.378298+0000", 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5586999999999998e-05 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.980 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:30.981 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.378306+0000", 2024-04-26T04:32:30.981 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6549999999999993e-06 2024-04-26T04:32:30.981 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.981 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.981 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:30.981 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478496+0000", 2024-04-26T04:32:30.981 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100189419 2024-04-26T04:32:30.981 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.981 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.981 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:30.981 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478509+0000", 2024-04-26T04:32:30.981 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2918999999999999e-05 2024-04-26T04:32:30.981 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.982 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.982 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:30.982 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.480025+0000", 2024-04-26T04:32:30.982 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0015167290000000001 2024-04-26T04:32:30.982 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:30.982 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:30.982 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:30.982 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.480081+0000", 2024-04-26T04:32:30.982 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.5784e-05 2024-04-26T04:32:30.982 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.982 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.982 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.982 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:30.983 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:30.983 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:30.984 INFO:tasks.rados.rados.0.smithi178.stdout:96: finishing write tid 4 to smithi17829308-21 2024-04-26T04:32:30.984 INFO:tasks.rados.rados.0.smithi178.stdout:96: oid 21 updating version 0 to 23 2024-04-26T04:32:30.984 INFO:tasks.rados.rados.0.smithi178.stdout:96: oid 21 updating version 23 to 25 2024-04-26T04:32:30.984 INFO:tasks.rados.rados.0.smithi178.stdout:96: oid 21 version 25 is already newer than 24 2024-04-26T04:32:30.984 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 21 v 25 (ObjNum 57 snap 1 seq_num 57) dirty exists 2024-04-26T04:32:30.984 INFO:tasks.rados.rados.0.smithi178.stdout:96: left oid 21 (ObjNum 57 snap 1 seq_num 57) 2024-04-26T04:32:30.985 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T04:32:30.986 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:30.980+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:31.048 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [2] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T04:32:31.048 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T04:32:31.048 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-26T04:32:31.049 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-26T04:32:31.068 INFO:tasks.rados.rados.0.smithi178.stdout:96: done (4 left) 2024-04-26T04:32:31.068 INFO:tasks.rados.rados.0.smithi178.stdout:105: done (3 left) 2024-04-26T04:32:31.068 INFO:tasks.rados.rados.0.smithi178.stdout:106: setattr oid 46 current snap is 1 2024-04-26T04:32:31.072 INFO:tasks.rados.rados.0.smithi178.stdout:107: snap_create 2024-04-26T04:32:31.085 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:31.080+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:31.100 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T04:32:31.112 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:31.112 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:31.112 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 0 2024-04-26T04:32:31.112 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:31.127 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T04:32:31.186 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:31.183+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:31.236 INFO:tasks.rados.rados.0.smithi178.stdout:107: done (4 left) 2024-04-26T04:32:31.236 INFO:tasks.rados.rados.0.smithi178.stdout:108: copy_from oid 38 from oid 4 current snap is 2 2024-04-26T04:32:31.236 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 18 current snap is 2 2024-04-26T04:32:31.236 INFO:tasks.rados.rados.0.smithi178.stdout:109: seq_num 62 ranges {294912=671744,966656=376832} 2024-04-26T04:32:31.247 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:31.248 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:31.250 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:31.250 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:31.250 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:32:31.250 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:32:31.250 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:31.251 INFO:tasks.rados.rados.0.smithi178.stdout:109: writing smithi17829308-18 from 294912 to 966656 tid 1 2024-04-26T04:32:31.258 INFO:tasks.rados.rados.0.smithi178.stdout:109: writing smithi17829308-18 from 966656 to 1343488 tid 2 2024-04-26T04:32:31.259 INFO:tasks.rados.rados.0.smithi178.stdout:110: snap_remove snap 1 2024-04-26T04:32:31.265 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T04:32:31.267 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:31.267 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:31.286 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:31.284+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:31.289 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:31.289 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:31.309 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-26T04:32:31.310 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:31.311 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:31.326 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:31.327 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:31.345 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:31.346 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:31.370 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:31.371 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:31.375 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:31.376 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 37, 2024-04-26T04:32:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "fsid": "12c27fbe-0161-48d0-88ec-42a226685437", 2024-04-26T04:32:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "created": "2024-04-26T04:31:56.451062+0000", 2024-04-26T04:32:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "modified": "2024-04-26T04:32:31.226628+0000", 2024-04-26T04:32:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "last_up_change": "2024-04-26T04:32:09.452464+0000", 2024-04-26T04:32:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "last_in_change": "2024-04-26T04:32:20.674579+0000", 2024-04-26T04:32:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-26T04:32:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_num": 5799936, 2024-04-26T04:32:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_set": [ 2024-04-26T04:32:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "pglog_hardlimit", 2024-04-26T04:32:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "purged_snapdirs", 2024-04-26T04:32:31.376 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_deletes", 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "sortbitwise" 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_version": 6, 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "full_ratio": 0.94999998807907104, 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_snapshot": "", 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_max": 3, 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd": 16, 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "require_min_compat_client": "luminous", 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "min_compat_client": "jewel", 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "require_osd_release": "squid", 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_crimson": false, 2024-04-26T04:32:31.377 INFO:teuthology.orchestra.run.smithi012.stdout: "pools": [ 2024-04-26T04:32:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 1, 2024-04-26T04:32:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": ".mgr", 2024-04-26T04:32:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-04-26T04:32:07.426875+0000", 2024-04-26T04:32:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T04:32:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool", 2024-04-26T04:32:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-04-26T04:32:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-04-26T04:32:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-04-26T04:32:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-04-26T04:32:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-04-26T04:32:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-04-26T04:32:31.378 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 1, 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 1, 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 1, 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 1, 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 1, 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-04-26T04:32:31.379 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-04-26T04:32:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-04-26T04:32:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "25", 2024-04-26T04:32:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-04-26T04:32:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-26T04:32:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-04-26T04:32:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-04-26T04:32:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-04-26T04:32:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 0, 2024-04-26T04:32:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 0, 2024-04-26T04:32:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-04-26T04:32:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-04-26T04:32:31.380 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-04-26T04:32:31.381 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-04-26T04:32:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-04-26T04:32:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-04-26T04:32:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-04-26T04:32:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-04-26T04:32:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-04-26T04:32:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-04-26T04:32:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-04-26T04:32:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T04:32:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-04-26T04:32:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-04-26T04:32:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-04-26T04:32:31.382 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "options": { 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_max": 32, 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_min": 1 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": {} 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "score_type": "Fair distribution", 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 15.380000114440918, 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 15.380000114440918, 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.12999999523162842, 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2, 2024-04-26T04:32:31.383 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2, 2024-04-26T04:32:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 1, 2024-04-26T04:32:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 1, 2024-04-26T04:32:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 1 2024-04-26T04:32:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-04-26T04:32:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "rbd", 2024-04-26T04:32:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-04-26T04:32:11.318957+0000", 2024-04-26T04:32:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T04:32:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool", 2024-04-26T04:32:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-04-26T04:32:31.384 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-04-26T04:32:31.385 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-04-26T04:32:31.385 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-04-26T04:32:31.385 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-04-26T04:32:31.385 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-04-26T04:32:31.385 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T04:32:31.385 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T04:32:31.385 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-04-26T04:32:31.385 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-04-26T04:32:31.385 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 8, 2024-04-26T04:32:31.385 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 8, 2024-04-26T04:32:31.385 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 8, 2024-04-26T04:32:31.385 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 8, 2024-04-26T04:32:31.386 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 8, 2024-04-26T04:32:31.386 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-04-26T04:32:31.386 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-04-26T04:32:31.386 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-04-26T04:32:31.386 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-04-26T04:32:31.386 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-04-26T04:32:31.386 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-04-26T04:32:31.386 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-04-26T04:32:31.386 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.386 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "27", 2024-04-26T04:32:31.387 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-04-26T04:32:31.387 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-26T04:32:31.387 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-04-26T04:32:31.387 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-04-26T04:32:31.387 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-04-26T04:32:31.387 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 0, 2024-04-26T04:32:31.387 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 0, 2024-04-26T04:32:31.387 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-04-26T04:32:31.387 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-04-26T04:32:31.387 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-04-26T04:32:31.387 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-04-26T04:32:31.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-04-26T04:32:31.387 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-04-26T04:32:31.388 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.389 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-04-26T04:32:31.389 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-04-26T04:32:31.389 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-04-26T04:32:31.389 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-04-26T04:32:31.389 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-04-26T04:32:31.389 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T04:32:31.389 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-04-26T04:32:31.389 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-04-26T04:32:31.389 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-04-26T04:32:31.390 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-04-26T04:32:31.390 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-04-26T04:32:31.390 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-04-26T04:32:31.390 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-04-26T04:32:31.390 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-04-26T04:32:31.390 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.390 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-04-26T04:32:31.390 INFO:teuthology.orchestra.run.smithi012.stdout: "score_type": "Fair distribution", 2024-04-26T04:32:31.390 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 4.0199999809265137, 2024-04-26T04:32:31.390 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 4.0199999809265137, 2024-04-26T04:32:31.390 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.56000000238418579, 2024-04-26T04:32:31.390 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2.25, 2024-04-26T04:32:31.390 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2.25, 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 1, 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 1, 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 1 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "unique_pool_0", 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-04-26T04:32:21.196824+0000", 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 3, 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 7, 2024-04-26T04:32:31.391 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 5, 2024-04-26T04:32:31.392 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 1, 2024-04-26T04:32:31.392 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-04-26T04:32:31.392 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-04-26T04:32:31.392 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T04:32:31.392 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T04:32:31.392 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-04-26T04:32:31.392 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-04-26T04:32:31.392 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 16, 2024-04-26T04:32:31.392 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 16, 2024-04-26T04:32:31.392 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 16, 2024-04-26T04:32:31.392 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 16, 2024-04-26T04:32:31.392 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 16, 2024-04-26T04:32:31.392 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-04-26T04:32:31.393 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-04-26T04:32:31.393 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-04-26T04:32:31.393 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-04-26T04:32:31.393 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-04-26T04:32:31.393 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-04-26T04:32:31.393 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-04-26T04:32:31.393 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.393 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "37", 2024-04-26T04:32:31.393 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-04-26T04:32:31.393 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-26T04:32:31.393 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-04-26T04:32:31.393 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-04-26T04:32:31.393 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-04-26T04:32:31.394 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 3, 2024-04-26T04:32:31.394 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 37, 2024-04-26T04:32:31.394 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-04-26T04:32:31.394 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-04-26T04:32:31.394 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-04-26T04:32:31.394 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-04-26T04:32:31.394 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-04-26T04:32:31.394 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-04-26T04:32:31.394 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-04-26T04:32:31.394 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-04-26T04:32:31.394 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-04-26T04:32:31.394 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-04-26T04:32:31.394 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-04-26T04:32:31.395 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T04:32:31.395 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T04:32:31.395 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T04:32:31.395 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-04-26T04:32:31.395 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-04-26T04:32:31.395 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "shecprofile", 2024-04-26T04:32:31.395 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-04-26T04:32:31.395 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-04-26T04:32:31.395 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.395 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-04-26T04:32:31.396 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-04-26T04:32:31.396 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-04-26T04:32:31.396 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-04-26T04:32:31.396 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-04-26T04:32:31.396 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T04:32:31.396 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-04-26T04:32:31.396 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-04-26T04:32:31.396 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 16384, 2024-04-26T04:32:31.396 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-04-26T04:32:31.396 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: "rados": {} 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: "osds": [ 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "55a4f254-2c2a-4007-9c52-b89df6fffb69", 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.397 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.398 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.398 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.398 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 21, 2024-04-26T04:32:31.398 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:32:31.398 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.398 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.398 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.398 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.398 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.398 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.398 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6816", 2024-04-26T04:32:31.398 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:32:31.398 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6817", 2024-04-26T04:32:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:32:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6818", 2024-04-26T04:32:31.399 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6819", 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6822", 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:32:31.400 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6823", 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6820", 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:32:31.401 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6821", 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6817/1754063127", 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6819/1754063127", 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6823/1754063127", 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6821/1754063127", 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.402 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "32db4f97-c8ac-481c-9e80-18fc91004298", 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 0, 2024-04-26T04:32:31.403 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6808", 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6809", 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.404 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6810", 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6811", 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.405 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.406 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.406 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.406 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6814", 2024-04-26T04:32:31.406 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:32:31.406 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.406 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.406 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.406 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6817", 2024-04-26T04:32:31.406 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:32:31.406 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.406 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6812", 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6813", 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.407 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.113:6809/3264326676", 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.113:6811/3264326676", 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.113:6817/3264326676", 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.113:6813/3264326676", 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "2cd26466-5d59-4599-ba3f-a78b0e3b681f", 2024-04-26T04:32:31.408 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 0, 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 0, 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 0, 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6800", 2024-04-26T04:32:31.409 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:32:31.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.410 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6802", 2024-04-26T04:32:31.410 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:32:31.410 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.410 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.410 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.410 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.410 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.410 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.410 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.410 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6804", 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6806", 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.411 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6812", 2024-04-26T04:32:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:32:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6814", 2024-04-26T04:32:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:32:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.412 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6808", 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6810", 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.137:6802/1454475448", 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.137:6806/1454475448", 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.137:6814/1454475448", 2024-04-26T04:32:31.413 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.137:6810/1454475448", 2024-04-26T04:32:31.414 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.414 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.414 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.414 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.414 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.414 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.414 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-04-26T04:32:31.414 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "0945017d-9525-49d6-aa67-6635acd94c3e", 2024-04-26T04:32:31.414 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.414 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.414 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.414 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.414 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 22, 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6803", 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.415 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.416 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6805", 2024-04-26T04:32:31.416 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:32:31.416 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.416 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.416 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.416 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.416 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.416 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6808", 2024-04-26T04:32:31.416 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:32:31.416 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.416 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6810", 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6816", 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.417 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6817", 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6813", 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.418 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6815", 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.178:6805/2898698985", 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.178:6810/2898698985", 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.178:6817/2898698985", 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.178:6815/2898698985", 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.419 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.420 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.420 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-04-26T04:32:31.420 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "38dc68bf-0a89-4900-9d35-24c70368447a", 2024-04-26T04:32:31.420 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.420 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.420 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.420 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.420 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.420 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.420 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 21, 2024-04-26T04:32:31.420 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:32:31.420 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.420 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6808", 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6809", 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.421 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6810", 2024-04-26T04:32:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:32:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6811", 2024-04-26T04:32:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:32:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.422 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.423 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.423 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.423 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6814", 2024-04-26T04:32:31.423 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:32:31.423 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.423 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.423 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.423 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6815", 2024-04-26T04:32:31.423 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6812", 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6813", 2024-04-26T04:32:31.424 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:32:31.425 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.425 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.425 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.425 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6809/3209840823", 2024-04-26T04:32:31.425 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6811/3209840823", 2024-04-26T04:32:31.425 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6815/3209840823", 2024-04-26T04:32:31.425 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6813/3209840823", 2024-04-26T04:32:31.425 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.425 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "e91cc8a1-57e1-4760-a329-8a60aa752a95", 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:32:31.426 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6800", 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6801", 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.427 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6802", 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6803", 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.428 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6806", 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6807", 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.429 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6804", 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6805", 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.430 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.431 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.113:6801/1475637773", 2024-04-26T04:32:31.431 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.113:6803/1475637773", 2024-04-26T04:32:31.431 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.113:6807/1475637773", 2024-04-26T04:32:31.431 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.113:6805/1475637773", 2024-04-26T04:32:31.431 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.431 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.431 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.431 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.431 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.431 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.431 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 6, 2024-04-26T04:32:31.431 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "a1322130-3391-432d-a1ee-f83297885029", 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 25, 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.432 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6816", 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6817", 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.433 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6818", 2024-04-26T04:32:31.434 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:32:31.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.434 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6819", 2024-04-26T04:32:31.434 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:32:31.434 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.434 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.434 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.434 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.434 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.434 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.434 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6822", 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6823", 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.435 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6820", 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6821", 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.137:6817/771338783", 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.137:6819/771338783", 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.137:6823/771338783", 2024-04-26T04:32:31.436 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.137:6821/771338783", 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 7, 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "02ee4ca2-9d48-4aff-9641-1823760c7a3b", 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.437 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 22, 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6826", 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6827", 2024-04-26T04:32:31.438 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6828", 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6829", 2024-04-26T04:32:31.439 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6832", 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6833", 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:32:31.440 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6830", 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6831", 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:32:31.441 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.178:6827/1017152077", 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.178:6829/1017152077", 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.178:6833/1017152077", 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.178:6831/1017152077", 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 8, 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "ea3d5f62-3d68-4824-8556-da8db36553fb", 2024-04-26T04:32:31.442 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 21, 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.443 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.444 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6802", 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6803", 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6806", 2024-04-26T04:32:31.445 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6807", 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6804", 2024-04-26T04:32:31.446 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6805", 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6801/2153129138", 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6803/2153129138", 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6807/2153129138", 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6805/2153129138", 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.447 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.448 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.448 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.448 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.448 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 9, 2024-04-26T04:32:31.448 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70", 2024-04-26T04:32:31.448 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.448 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.448 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.448 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.448 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.448 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.448 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:32:31.448 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 0, 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6824", 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6825", 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:32:31.449 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6826", 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6827", 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:32:31.450 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.451 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.451 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.451 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.451 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6830", 2024-04-26T04:32:31.451 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:32:31.451 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.451 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.451 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.451 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6831", 2024-04-26T04:32:31.451 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6828", 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6829", 2024-04-26T04:32:31.452 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.113:6825/359931551", 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.113:6827/359931551", 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.113:6831/359931551", 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.113:6829/359931551", 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.453 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 10, 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "8c4f8df5-465a-486e-9413-1de14e33c119", 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 21, 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 0, 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.454 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6801", 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6803", 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.455 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6805", 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6807", 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.456 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6813", 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6815", 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.457 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6809", 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6811", 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.137:6803/1533480861", 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.137:6807/1533480861", 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.137:6815/1533480861", 2024-04-26T04:32:31.458 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.137:6811/1533480861", 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 11, 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "6fe54c2d-0b95-4235-8c22-51230cfaa591", 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.459 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 22, 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6818", 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6819", 2024-04-26T04:32:31.460 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6820", 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6821", 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:32:31.461 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6824", 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6825", 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:32:31.462 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6822", 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6823", 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.463 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.178:6819/1683682957", 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.178:6821/1683682957", 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.178:6825/1683682957", 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.178:6823/1683682957", 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 12, 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "5ddd0f05-fb78-43bb-95ec-3fc7985603b3", 2024-04-26T04:32:31.464 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 0, 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.465 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6824", 2024-04-26T04:32:31.466 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:32:31.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.466 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.466 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6825", 2024-04-26T04:32:31.466 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:32:31.466 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.466 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.466 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.466 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.466 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.466 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.466 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6826", 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6827", 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.467 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6830", 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6831", 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.468 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6828", 2024-04-26T04:32:31.469 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:32:31.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.469 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.469 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.469 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6829", 2024-04-26T04:32:31.469 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:32:31.469 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.469 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.469 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.469 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6825/2509311840", 2024-04-26T04:32:31.469 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6827/2509311840", 2024-04-26T04:32:31.469 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6831/2509311840", 2024-04-26T04:32:31.469 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6829/2509311840", 2024-04-26T04:32:31.470 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.470 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.470 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.470 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.470 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.470 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.470 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 13, 2024-04-26T04:32:31.470 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "37eb42c1-f29e-49f5-b8f6-035178038e5d", 2024-04-26T04:32:31.470 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.470 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.470 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.470 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.470 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 31, 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6815", 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.471 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6816", 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6818", 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.472 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6819", 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6822", 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.473 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6823", 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6820", 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.474 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6821", 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.113:6816/3716239663", 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.113:6819/3716239663", 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.113:6823/3716239663", 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.113:6821/3716239663", 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.475 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 14, 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "b2267367-a857-4a5c-a8d1-6a9b23c748f6", 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.476 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6824", 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6825", 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.477 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6826", 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6827", 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.478 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6830", 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6831", 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.479 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6828", 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6829", 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.137:6825/1454956132", 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.137:6827/1454956132", 2024-04-26T04:32:31.480 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.137:6831/1454956132", 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.137:6829/1454956132", 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 15, 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "9933b5aa-e091-43b1-b788-5b79695bab11", 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:32:31.481 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 21, 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6802", 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.482 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6804", 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6806", 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.483 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6807", 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6812", 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.484 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6814", 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6809", 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:32:31.485 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6811", 2024-04-26T04:32:31.486 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:32:31.487 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.487 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.487 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.487 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.178:6804/314165740", 2024-04-26T04:32:31.487 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.178:6807/314165740", 2024-04-26T04:32:31.487 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.178:6814/314165740", 2024-04-26T04:32:31.487 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.178:6811/314165740", 2024-04-26T04:32:31.487 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:32:31.487 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:32:31.487 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:32:31.487 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:31.487 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.487 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_xinfo": [ 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.305255+0000", 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-04-26T04:32:31.488 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.420004+0000", 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.489 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 65536, 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.396744+0000", 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.471304+0000", 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.490 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.276649+0000", 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.491 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.349822+0000", 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 6, 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.396513+0000", 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.492 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 7, 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.438157+0000", 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 8, 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.493 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.166681+0000", 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 9, 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.532725+0000", 2024-04-26T04:32:31.494 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 10, 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.242376+0000", 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 11, 2024-04-26T04:32:31.495 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.328959+0000", 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 12, 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.496 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.440612+0000", 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 13, 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.485079+0000", 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.497 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 14, 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.290530+0000", 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 15, 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:32:31.498 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.232637+0000", 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap": [], 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_items": [], 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_primaries": [], 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_temp": [], 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_temp": [], 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklist": {}, 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "range_blocklist": {}, 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profiles": { 2024-04-26T04:32:31.499 INFO:teuthology.orchestra.run.smithi012.stdout: "default": { 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-failure-domain": "osd", 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "k": "2", 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "m": "1", 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin": "jerasure", 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "technique": "reed_sol_van" 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "shecprofile": { 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "c": "2", 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-device-class": "", 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-failure-domain": "osd", 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-num-failure-domains": "0", 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-osds-per-failure-domain": "0", 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-root": "default", 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "k": "4", 2024-04-26T04:32:31.500 INFO:teuthology.orchestra.run.smithi012.stdout: "m": "3", 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "shecprofile", 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin": "shec", 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: "technique": "multiple" 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps_queue": [], 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: "new_removed_snaps": [], 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: "new_purged_snaps": [], 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_node_flags": {}, 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: "device_class_flags": {}, 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode": { 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_enabled": false, 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_bucket_count": 0, 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: "degraded_stretch_mode": 0, 2024-04-26T04:32:31.501 INFO:teuthology.orchestra.run.smithi012.stdout: "recovering_stretch_mode": 0, 2024-04-26T04:32:31.502 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_bucket": 0 2024-04-26T04:32:31.502 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:31.502 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:31.503 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:31.504 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:31.505 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:31.505 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:32:31.505 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:32:31.505 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:32:31.505 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.505 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-26T04:32:31.505 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 21.921034758000001, 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.261775152, 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 1 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.506 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471107+0000", 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4540000000000001e-06 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.507 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471120+0000", 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3217999999999999e-05 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471147+0000", 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6461e-05 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471188+0000", 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1539000000000003e-05 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.508 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.472842+0000", 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016540700000000001 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for peered", 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.472873+0000", 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0300000000000001e-05 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.729650+0000", 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2567770170000001 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.509 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.729957+0000", 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00030700500000000001 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.730324+0000", 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00036723099999999998 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732599+0000", 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0022751260000000001 2024-04-26T04:32:31.510 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732706+0000", 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010741 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732804+0000", 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7348000000000004e-05 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732880+0000", 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5972999999999999e-05 2024-04-26T04:32:31.511 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708719+0000", 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 14.683419776000001, 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60283476700000005, 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:31.512 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 115 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708719+0000", 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708719+0000", 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.513 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708720+0000", 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6700000000000003e-07 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708854+0000", 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013397699999999999 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708866+0000", 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1875000000000001e-05 2024-04-26T04:32:31.514 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708874+0000", 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0600000000000008e-06 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.818243+0000", 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10936897900000001 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.818373+0000", 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012997700000000001 2024-04-26T04:32:31.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.818585+0000", 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021234699999999999 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820901+0000", 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0023151669999999999 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311251+0000", 2024-04-26T04:32:31.516 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.49035071699999999 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311421+0000", 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016933399999999999 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311554+0000", 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013366699999999999 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.518 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.518 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:31.518 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708888+0000", 2024-04-26T04:32:31.518 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 14.683251031999999, 2024-04-26T04:32:31.518 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60310785300000003, 2024-04-26T04:32:31.518 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.518 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.518 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.518 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.518 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:31.518 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:31.518 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 116 2024-04-26T04:32:31.518 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708888+0000", 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708888+0000", 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.519 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708889+0000", 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8700000000000005e-07 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708894+0000", 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.446e-06 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708904+0000", 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7059999999999999e-06 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.520 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708912+0000", 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3730000000000004e-06 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.819534+0000", 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.110621994 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.819754+0000", 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00022034999999999999 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.521 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.522 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:31.522 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820112+0000", 2024-04-26T04:32:31.522 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00035776299999999998 2024-04-26T04:32:31.522 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.522 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.522 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:31.522 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821816+0000", 2024-04-26T04:32:31.522 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001704346 2024-04-26T04:32:31.522 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.522 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.522 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:31.522 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311719+0000", 2024-04-26T04:32:31.522 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.48990303600000001 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311876+0000", 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015653800000000001 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311996+0000", 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012051400000000001 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.523 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 14.683216827000001, 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60751630199999995, 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 117 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.524 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708923+0000", 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3300000000000002e-07 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.525 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708929+0000", 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.9959999999999999e-06 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708941+0000", 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2085e-05 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708948+0000", 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.8170000000000001e-06 2024-04-26T04:32:31.526 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820967+0000", 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.112019008 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821107+0000", 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000139825 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821309+0000", 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020262899999999999 2024-04-26T04:32:31.527 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.822530+0000", 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0012206129999999999 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316288+0000", 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.49375772400000001 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316375+0000", 2024-04-26T04:32:31.528 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7312999999999996e-05 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316439+0000", 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3658999999999995e-05 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739091+0000", 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.653047757, 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51402278300000004, 2024-04-26T04:32:31.529 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 158 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739091+0000", 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.530 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739091+0000", 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739092+0000", 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.99e-07 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739179+0000", 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6510999999999999e-05 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.531 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739190+0000", 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1629e-05 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739201+0000", 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0739e-05 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.743817+0000", 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0046161170000000003 2024-04-26T04:32:31.532 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.743935+0000", 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000117661 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.744145+0000", 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021043900000000001 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.746703+0000", 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0025576919999999999 2024-04-26T04:32:31.533 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252837+0000", 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.506133783 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252994+0000", 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015751200000000001 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253114+0000", 2024-04-26T04:32:31.534 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012000100000000001 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739215+0000", 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.652924585999999, 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51429833800000002, 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:31.535 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 159 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739215+0000", 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739215+0000", 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.536 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739215+0000", 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8599999999999997e-07 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739222+0000", 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.1929999999999998e-06 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739235+0000", 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3815000000000001e-05 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.537 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739245+0000", 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.0929999999999992e-06 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.744841+0000", 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0055962429999999999 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.744959+0000", 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00011796200000000001 2024-04-26T04:32:31.538 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.745168+0000", 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020875099999999999 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.747905+0000", 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0027373330000000002 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253250+0000", 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.50534521399999999 2024-04-26T04:32:31.539 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253403+0000", 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015304899999999999 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253513+0000", 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000109899 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.540 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:160 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739257+0000", 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.652882599, 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51462529499999998, 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 160 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.541 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739257+0000", 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739257+0000", 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739257+0000", 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3600000000000003e-07 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.542 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739264+0000", 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.2360000000000001e-06 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739274+0000", 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.9960000000000005e-06 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739281+0000", 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.1300000000000003e-06 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.543 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.544 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.745763+0000", 2024-04-26T04:32:31.544 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0064821219999999999 2024-04-26T04:32:31.544 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.544 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.544 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.544 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.745879+0000", 2024-04-26T04:32:31.544 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000116348 2024-04-26T04:32:31.544 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.544 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.544 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:31.544 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.746176+0000", 2024-04-26T04:32:31.544 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00029663299999999998 2024-04-26T04:32:31.544 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.046853+0000", 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30067709799999998 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253632+0000", 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20677860000000001 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253776+0000", 2024-04-26T04:32:31.545 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014417 2024-04-26T04:32:31.546 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.546 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.546 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.546 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.253882+0000", 2024-04-26T04:32:31.546 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000106226 2024-04-26T04:32:31.546 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.546 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.546 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.546 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.546 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.546 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:161 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:31.546 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:32:31.546 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.652847471999999, 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51490446599999995, 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 161 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:32:31.547 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6599999999999996e-07 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739378+0000", 2024-04-26T04:32:31.548 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.5968000000000001e-05 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739390+0000", 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1595e-05 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739398+0000", 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0029999999999994e-06 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.746760+0000", 2024-04-26T04:32:31.549 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0073621420000000003 2024-04-26T04:32:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.746876+0000", 2024-04-26T04:32:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000115789 2024-04-26T04:32:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.747088+0000", 2024-04-26T04:32:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021234800000000001 2024-04-26T04:32:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.550 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252667+0000", 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.50557901500000002 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254031+0000", 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001363518 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254124+0000", 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3376999999999997e-05 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.551 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254196+0000", 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2045000000000006e-05 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739429+0000", 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.652710289, 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51514356699999997, 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.552 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 162 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739429+0000", 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.553 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739429+0000", 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739430+0000", 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8999999999999995e-07 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739435+0000", 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.5969999999999999e-06 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.554 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739446+0000", 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0535e-05 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739453+0000", 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.4800000000000004e-06 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.943028+0000", 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20357512 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.555 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.943276+0000", 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024770199999999998 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.943640+0000", 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00036409399999999997 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252675+0000", 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30903449199999999 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.556 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254318+0000", 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016429179999999999 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254463+0000", 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000145623 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254573+0000", 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010921600000000001 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.557 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.652673628000001, 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.515346639, 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 163 2024-04-26T04:32:31.558 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.559 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.1699999999999997e-07 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739471+0000", 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.9219999999999996e-06 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739487+0000", 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5435999999999999e-05 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.560 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739494+0000", 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2899999999999997e-06 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.944605+0000", 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20511091200000001 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.944825+0000", 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00022035499999999999 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.561 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.945203+0000", 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00037760499999999999 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252681+0000", 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.307478311 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254680+0000", 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0019988129999999999 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.562 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254756+0000", 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5914999999999994e-05 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254812+0000", 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.6363000000000002e-05 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.563 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.652633491, 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.61583993999999997, 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 164 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.564 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0399999999999995e-07 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.565 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739594+0000", 2024-04-26T04:32:31.566 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7046000000000007e-05 2024-04-26T04:32:31.566 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.566 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.566 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.566 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739605+0000", 2024-04-26T04:32:31.566 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1576000000000001e-05 2024-04-26T04:32:31.566 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.566 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.566 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.566 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739613+0000", 2024-04-26T04:32:31.566 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1459999999999996e-06 2024-04-26T04:32:31.566 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.566 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.046981+0000", 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30736738000000002 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.047229+0000", 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024846999999999998 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.047619+0000", 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00038963799999999999 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.567 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254219+0000", 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20659982700000001 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355048+0000", 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100829484 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355225+0000", 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017717399999999999 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.568 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355346+0000", 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000120495 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.652511976, 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.71903434899999996, 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.569 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 165 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.570 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739628+0000", 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7000000000000004e-07 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739645+0000", 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6901999999999998e-05 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.571 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739656+0000", 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0556e-05 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739667+0000", 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1535000000000001e-05 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355474+0000", 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.61580721299999996 2024-04-26T04:32:31.572 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355612+0000", 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000137687 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355816+0000", 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020451900000000001 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.357276+0000", 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0014599179999999999 2024-04-26T04:32:31.573 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458392+0000", 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101115921 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458547+0000", 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015461199999999999 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458662+0000", 2024-04-26T04:32:31.574 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000114716 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:31 3.es0 3:7950d065:::smithi17829308-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.211346+0000", 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.1807928289999996, 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80728777299999999, 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.575 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 31 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211346+0000", 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211346+0000", 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.576 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211350+0000", 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.3400000000000002e-06 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211978+0000", 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00062828600000000004 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212008+0000", 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0287999999999999e-05 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.577 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212024+0000", 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5316e-05 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312296+0000", 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100272797 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312539+0000", 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000242542 2024-04-26T04:32:31.578 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.313858+0000", 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001319312 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319757+0000", 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.005898927 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018515+0000", 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69875785199999996 2024-04-26T04:32:31.579 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018634+0000", 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000119113 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:32 3.es0 3:7950d065:::smithi17829308-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.212058+0000", 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.1800808209999998, 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80687460799999999, 2024-04-26T04:32:31.580 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 32 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212058+0000", 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.581 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212058+0000", 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212060+0000", 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2130000000000001e-06 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212065+0000", 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.5260000000000001e-06 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.582 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212084+0000", 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8399000000000001e-05 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212114+0000", 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0598000000000001e-05 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.314776+0000", 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10266165200000001 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.583 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.315053+0000", 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00027713999999999999 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316307+0000", 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00125442 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321719+0000", 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.005411463 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.584 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.585 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.585 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018816+0000", 2024-04-26T04:32:31.585 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69709755699999998 2024-04-26T04:32:31.585 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.585 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.585 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.585 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018933+0000", 2024-04-26T04:32:31.585 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00011663999999999999 2024-04-26T04:32:31.585 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.585 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.585 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.585 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.585 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:43 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~245760,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.217138+0000", 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.1750007819999997, 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80275945999999998, 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 43 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.586 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217138+0000", 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217138+0000", 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217141+0000", 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3499999999999999e-06 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.587 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217563+0000", 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00042241199999999999 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217591+0000", 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7501999999999999e-05 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217605+0000", 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3866000000000001e-05 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.588 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.317118+0000", 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.099513764000000005 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.317307+0000", 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018885199999999999 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.318924+0000", 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001616691 2024-04-26T04:32:31.589 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321728+0000", 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0028038249999999998 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.019782+0000", 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69805386400000002 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.019898+0000", 2024-04-26T04:32:31.590 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000116334 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:44 3.es0 3:7dd262f9:::smithi17829308-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.217633+0000", 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.174506321, 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80280496999999995, 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:31.591 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 44 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217633+0000", 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217633+0000", 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.592 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217634+0000", 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.06e-06 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217640+0000", 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.614e-06 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217654+0000", 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4279e-05 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.593 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217662+0000", 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2959999999999999e-06 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319858+0000", 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102196092 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319985+0000", 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012646000000000001 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.594 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.320976+0000", 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00099148399999999994 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.322066+0000", 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001090246 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.020239+0000", 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69817257799999999 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.595 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.020438+0000", 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00019886099999999999 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:267 3.2s0 3:4497087f:::smithi17829308-22:head [read 0~360448,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.131109+0000", 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.261030203, 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70452207899999997, 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.596 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 267 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131109+0000", 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.597 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131109+0000", 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131111+0000", 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9700000000000002e-06 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131119+0000", 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4600000000000003e-06 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.598 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131136+0000", 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6820000000000002e-05 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131146+0000", 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.2499999999999995e-06 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131232+0000", 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6907e-05 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.599 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131352+0000", 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000119164 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.835631+0000", 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70427950800000005 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:268 3.2s0 3:4497087f:::smithi17829308-22:head [read 0~360448,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:31.600 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.2609706570000001, 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70524462600000004, 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 268 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.601 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3799999999999997e-07 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.602 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131173+0000", 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8929999999999998e-06 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131188+0000", 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5208000000000001e-05 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131197+0000", 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1950000000000003e-06 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.603 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131969+0000", 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00077229900000000005 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.132116+0000", 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014686300000000001 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.836413+0000", 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70429752999999995 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.604 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:269 3.2s0 3:4497087f:::smithi17829308-22:head [read 0~360448,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.131236+0000", 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.2609027660000001, 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70573893899999995, 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 269 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.605 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131236+0000", 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131236+0000", 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131237+0000", 2024-04-26T04:32:31.606 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.9800000000000003e-07 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131241+0000", 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.2239999999999997e-06 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131250+0000", 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.0629999999999995e-06 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231335+0000", 2024-04-26T04:32:31.607 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10008484400000001 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231436+0000", 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000100435 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231576+0000", 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014048000000000001 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.836975+0000", 2024-04-26T04:32:31.608 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.605399295 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:273 3.2s0 3:466bd692:::smithi17829308-25:head [delete] snapc 2=[2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.231373+0000", 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.1607658409999999, 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60595961600000003, 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:31.609 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 273 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231373+0000", 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231373+0000", 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.610 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231375+0000", 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.539e-06 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231383+0000", 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5660000000000001e-06 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231394+0000", 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1739000000000001e-05 2024-04-26T04:32:31.611 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231405+0000", 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0266e-05 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231995+0000", 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00059051299999999998 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.232097+0000", 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010197899999999999 2024-04-26T04:32:31.612 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.233122+0000", 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00102451 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.234316+0000", 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0011942019999999999 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.837219+0000", 2024-04-26T04:32:31.613 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60290286500000001 2024-04-26T04:32:31.614 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:31.614 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:31.614 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:31.614 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.837333+0000", 2024-04-26T04:32:31.614 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000114437 2024-04-26T04:32:31.614 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.614 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.614 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.614 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:31.614 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:31.614 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:31.616 INFO:tasks.thrashosds.thrasher:j is {'epoch': 37, 'fsid': '12c27fbe-0161-48d0-88ec-42a226685437', 'created': '2024-04-26T04:31:56.451062+0000', 'modified': '2024-04-26T04:32:31.226628+0000', 'last_up_change': '2024-04-26T04:32:09.452464+0000', 'last_in_change': '2024-04-26T04:32:20.674579+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T04:32:07.426875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-26T04:32:11.318957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.019999980926514, 'score_stable': 4.019999980926514, 'optimal_score': 0.5600000023841858, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-04-26T04:32:21.196824+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 37, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '55a4f254-2c2a-4007-9c52-b89df6fffb69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6816', 'nonce': 1754063127}, {'type': 'v1', 'addr': '172.21.15.12:6817', 'nonce': 1754063127}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6818', 'nonce': 1754063127}, {'type': 'v1', 'addr': '172.21.15.12:6819', 'nonce': 1754063127}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6822', 'nonce': 1754063127}, {'type': 'v1', 'addr': '172.21.15.12:6823', 'nonce': 1754063127}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6820', 'nonce': 1754063127}, {'type': 'v1', 'addr': '172.21.15.12:6821', 'nonce': 1754063127}]}, 'public_addr': '172.21.15.12:6817/1754063127', 'cluster_addr': '172.21.15.12:6819/1754063127', 'heartbeat_back_addr': '172.21.15.12:6823/1754063127', 'heartbeat_front_addr': '172.21.15.12:6821/1754063127', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '32db4f97-c8ac-481c-9e80-18fc91004298', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6808', 'nonce': 3264326676}, {'type': 'v1', 'addr': '172.21.15.113:6809', 'nonce': 3264326676}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6810', 'nonce': 3264326676}, {'type': 'v1', 'addr': '172.21.15.113:6811', 'nonce': 3264326676}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6814', 'nonce': 3264326676}, {'type': 'v1', 'addr': '172.21.15.113:6817', 'nonce': 3264326676}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6812', 'nonce': 3264326676}, {'type': 'v1', 'addr': '172.21.15.113:6813', 'nonce': 3264326676}]}, 'public_addr': '172.21.15.113:6809/3264326676', 'cluster_addr': '172.21.15.113:6811/3264326676', 'heartbeat_back_addr': '172.21.15.113:6817/3264326676', 'heartbeat_front_addr': '172.21.15.113:6813/3264326676', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2cd26466-5d59-4599-ba3f-a78b0e3b681f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6800', 'nonce': 1454475448}, {'type': 'v1', 'addr': '172.21.15.137:6802', 'nonce': 1454475448}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6804', 'nonce': 1454475448}, {'type': 'v1', 'addr': '172.21.15.137:6806', 'nonce': 1454475448}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6812', 'nonce': 1454475448}, {'type': 'v1', 'addr': '172.21.15.137:6814', 'nonce': 1454475448}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6808', 'nonce': 1454475448}, {'type': 'v1', 'addr': '172.21.15.137:6810', 'nonce': 1454475448}]}, 'public_addr': '172.21.15.137:6802/1454475448', 'cluster_addr': '172.21.15.137:6806/1454475448', 'heartbeat_back_addr': '172.21.15.137:6814/1454475448', 'heartbeat_front_addr': '172.21.15.137:6810/1454475448', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0945017d-9525-49d6-aa67-6635acd94c3e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6803', 'nonce': 2898698985}, {'type': 'v1', 'addr': '172.21.15.178:6805', 'nonce': 2898698985}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6808', 'nonce': 2898698985}, {'type': 'v1', 'addr': '172.21.15.178:6810', 'nonce': 2898698985}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6816', 'nonce': 2898698985}, {'type': 'v1', 'addr': '172.21.15.178:6817', 'nonce': 2898698985}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6813', 'nonce': 2898698985}, {'type': 'v1', 'addr': '172.21.15.178:6815', 'nonce': 2898698985}]}, 'public_addr': '172.21.15.178:6805/2898698985', 'cluster_addr': '172.21.15.178:6810/2898698985', 'heartbeat_back_addr': '172.21.15.178:6817/2898698985', 'heartbeat_front_addr': '172.21.15.178:6815/2898698985', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '38dc68bf-0a89-4900-9d35-24c70368447a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6808', 'nonce': 3209840823}, {'type': 'v1', 'addr': '172.21.15.12:6809', 'nonce': 3209840823}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6810', 'nonce': 3209840823}, {'type': 'v1', 'addr': '172.21.15.12:6811', 'nonce': 3209840823}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6814', 'nonce': 3209840823}, {'type': 'v1', 'addr': '172.21.15.12:6815', 'nonce': 3209840823}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6812', 'nonce': 3209840823}, {'type': 'v1', 'addr': '172.21.15.12:6813', 'nonce': 3209840823}]}, 'public_addr': '172.21.15.12:6809/3209840823', 'cluster_addr': '172.21.15.12:6811/3209840823', 'heartbeat_back_addr': '172.21.15.12:6815/3209840823', 'heartbeat_front_addr': '172.21.15.12:6813/3209840823', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e91cc8a1-57e1-4760-a329-8a60aa752a95', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6800', 'nonce': 1475637773}, {'type': 'v1', 'addr': '172.21.15.113:6801', 'nonce': 1475637773}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6802', 'nonce': 1475637773}, {'type': 'v1', 'addr': '172.21.15.113:6803', 'nonce': 1475637773}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6806', 'nonce': 1475637773}, {'type': 'v1', 'addr': '172.21.15.113:6807', 'nonce': 1475637773}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6804', 'nonce': 1475637773}, {'type': 'v1', 'addr': '172.21.15.113:6805', 'nonce': 1475637773}]}, 'public_addr': '172.21.15.113:6801/1475637773', 'cluster_addr': '172.21.15.113:6803/1475637773', 'heartbeat_back_addr': '172.21.15.113:6807/1475637773', 'heartbeat_front_addr': '172.21.15.113:6805/1475637773', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'a1322130-3391-432d-a1ee-f83297885029', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6816', 'nonce': 771338783}, {'type': 'v1', 'addr': '172.21.15.137:6817', 'nonce': 771338783}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6818', 'nonce': 771338783}, {'type': 'v1', 'addr': '172.21.15.137:6819', 'nonce': 771338783}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6822', 'nonce': 771338783}, {'type': 'v1', 'addr': '172.21.15.137:6823', 'nonce': 771338783}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6820', 'nonce': 771338783}, {'type': 'v1', 'addr': '172.21.15.137:6821', 'nonce': 771338783}]}, 'public_addr': '172.21.15.137:6817/771338783', 'cluster_addr': '172.21.15.137:6819/771338783', 'heartbeat_back_addr': '172.21.15.137:6823/771338783', 'heartbeat_front_addr': '172.21.15.137:6821/771338783', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '02ee4ca2-9d48-4aff-9641-1823760c7a3b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6826', 'nonce': 1017152077}, {'type': 'v1', 'addr': '172.21.15.178:6827', 'nonce': 1017152077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6828', 'nonce': 1017152077}, {'type': 'v1', 'addr': '172.21.15.178:6829', 'nonce': 1017152077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 1017152077}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 1017152077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6830', 'nonce': 1017152077}, {'type': 'v1', 'addr': '172.21.15.178:6831', 'nonce': 1017152077}]}, 'public_addr': '172.21.15.178:6827/1017152077', 'cluster_addr': '172.21.15.178:6829/1017152077', 'heartbeat_back_addr': '172.21.15.178:6833/1017152077', 'heartbeat_front_addr': '172.21.15.178:6831/1017152077', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': 'ea3d5f62-3d68-4824-8556-da8db36553fb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6800', 'nonce': 2153129138}, {'type': 'v1', 'addr': '172.21.15.12:6801', 'nonce': 2153129138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6802', 'nonce': 2153129138}, {'type': 'v1', 'addr': '172.21.15.12:6803', 'nonce': 2153129138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6806', 'nonce': 2153129138}, {'type': 'v1', 'addr': '172.21.15.12:6807', 'nonce': 2153129138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6804', 'nonce': 2153129138}, {'type': 'v1', 'addr': '172.21.15.12:6805', 'nonce': 2153129138}]}, 'public_addr': '172.21.15.12:6801/2153129138', 'cluster_addr': '172.21.15.12:6803/2153129138', 'heartbeat_back_addr': '172.21.15.12:6807/2153129138', 'heartbeat_front_addr': '172.21.15.12:6805/2153129138', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': 'e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6824', 'nonce': 359931551}, {'type': 'v1', 'addr': '172.21.15.113:6825', 'nonce': 359931551}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6826', 'nonce': 359931551}, {'type': 'v1', 'addr': '172.21.15.113:6827', 'nonce': 359931551}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6830', 'nonce': 359931551}, {'type': 'v1', 'addr': '172.21.15.113:6831', 'nonce': 359931551}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6828', 'nonce': 359931551}, {'type': 'v1', 'addr': '172.21.15.113:6829', 'nonce': 359931551}]}, 'public_addr': '172.21.15.113:6825/359931551', 'cluster_addr': '172.21.15.113:6827/359931551', 'heartbeat_back_addr': '172.21.15.113:6831/359931551', 'heartbeat_front_addr': '172.21.15.113:6829/359931551', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '8c4f8df5-465a-486e-9413-1de14e33c119', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6801', 'nonce': 1533480861}, {'type': 'v1', 'addr': '172.21.15.137:6803', 'nonce': 1533480861}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6805', 'nonce': 1533480861}, {'type': 'v1', 'addr': '172.21.15.137:6807', 'nonce': 1533480861}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6813', 'nonce': 1533480861}, {'type': 'v1', 'addr': '172.21.15.137:6815', 'nonce': 1533480861}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6809', 'nonce': 1533480861}, {'type': 'v1', 'addr': '172.21.15.137:6811', 'nonce': 1533480861}]}, 'public_addr': '172.21.15.137:6803/1533480861', 'cluster_addr': '172.21.15.137:6807/1533480861', 'heartbeat_back_addr': '172.21.15.137:6815/1533480861', 'heartbeat_front_addr': '172.21.15.137:6811/1533480861', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '6fe54c2d-0b95-4235-8c22-51230cfaa591', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6818', 'nonce': 1683682957}, {'type': 'v1', 'addr': '172.21.15.178:6819', 'nonce': 1683682957}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6820', 'nonce': 1683682957}, {'type': 'v1', 'addr': '172.21.15.178:6821', 'nonce': 1683682957}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6824', 'nonce': 1683682957}, {'type': 'v1', 'addr': '172.21.15.178:6825', 'nonce': 1683682957}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6822', 'nonce': 1683682957}, {'type': 'v1', 'addr': '172.21.15.178:6823', 'nonce': 1683682957}]}, 'public_addr': '172.21.15.178:6819/1683682957', 'cluster_addr': '172.21.15.178:6821/1683682957', 'heartbeat_back_addr': '172.21.15.178:6825/1683682957', 'heartbeat_front_addr': '172.21.15.178:6823/1683682957', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '5ddd0f05-fb78-43bb-95ec-3fc7985603b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6824', 'nonce': 2509311840}, {'type': 'v1', 'addr': '172.21.15.12:6825', 'nonce': 2509311840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6826', 'nonce': 2509311840}, {'type': 'v1', 'addr': '172.21.15.12:6827', 'nonce': 2509311840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6830', 'nonce': 2509311840}, {'type': 'v1', 'addr': '172.21.15.12:6831', 'nonce': 2509311840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6828', 'nonce': 2509311840}, {'type': 'v1', 'addr': '172.21.15.12:6829', 'nonce': 2509311840}]}, 'public_addr': '172.21.15.12:6825/2509311840', 'cluster_addr': '172.21.15.12:6827/2509311840', 'heartbeat_back_addr': '172.21.15.12:6831/2509311840', 'heartbeat_front_addr': '172.21.15.12:6829/2509311840', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '37eb42c1-f29e-49f5-b8f6-035178038e5d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6815', 'nonce': 3716239663}, {'type': 'v1', 'addr': '172.21.15.113:6816', 'nonce': 3716239663}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6818', 'nonce': 3716239663}, {'type': 'v1', 'addr': '172.21.15.113:6819', 'nonce': 3716239663}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6822', 'nonce': 3716239663}, {'type': 'v1', 'addr': '172.21.15.113:6823', 'nonce': 3716239663}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6820', 'nonce': 3716239663}, {'type': 'v1', 'addr': '172.21.15.113:6821', 'nonce': 3716239663}]}, 'public_addr': '172.21.15.113:6816/3716239663', 'cluster_addr': '172.21.15.113:6819/3716239663', 'heartbeat_back_addr': '172.21.15.113:6823/3716239663', 'heartbeat_front_addr': '172.21.15.113:6821/3716239663', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': 'b2267367-a857-4a5c-a8d1-6a9b23c748f6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6824', 'nonce': 1454956132}, {'type': 'v1', 'addr': '172.21.15.137:6825', 'nonce': 1454956132}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6826', 'nonce': 1454956132}, {'type': 'v1', 'addr': '172.21.15.137:6827', 'nonce': 1454956132}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6830', 'nonce': 1454956132}, {'type': 'v1', 'addr': '172.21.15.137:6831', 'nonce': 1454956132}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6828', 'nonce': 1454956132}, {'type': 'v1', 'addr': '172.21.15.137:6829', 'nonce': 1454956132}]}, 'public_addr': '172.21.15.137:6825/1454956132', 'cluster_addr': '172.21.15.137:6827/1454956132', 'heartbeat_back_addr': '172.21.15.137:6831/1454956132', 'heartbeat_front_addr': '172.21.15.137:6829/1454956132', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '9933b5aa-e091-43b1-b788-5b79695bab11', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6802', 'nonce': 314165740}, {'type': 'v1', 'addr': '172.21.15.178:6804', 'nonce': 314165740}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6806', 'nonce': 314165740}, {'type': 'v1', 'addr': '172.21.15.178:6807', 'nonce': 314165740}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6812', 'nonce': 314165740}, {'type': 'v1', 'addr': '172.21.15.178:6814', 'nonce': 314165740}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6809', 'nonce': 314165740}, {'type': 'v1', 'addr': '172.21.15.178:6811', 'nonce': 314165740}]}, 'public_addr': '172.21.15.178:6804/314165740', 'cluster_addr': '172.21.15.178:6807/314165740', 'heartbeat_back_addr': '172.21.15.178:6814/314165740', 'heartbeat_front_addr': '172.21.15.178:6811/314165740', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.305255+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.420004+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.396744+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.471304+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.276649+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.349822+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.396513+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.438157+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.166681+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.532725+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.242376+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.328959+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.440612+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.485079+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.290530+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.232637+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-26T04:32:31.616 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:32:31.616 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:31.617 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T04:32:31.617 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:31.498+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:31.617 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:31.618 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:31.618 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:31.619 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:31.619 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:31.619 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:31.620 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:31.620 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:31.620 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:31.620 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:31.621 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:31.621 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:31.621 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:31.622 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:31.622 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:31.623 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:31.616+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:31.720 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:31.718+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:31.750 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:31.750 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:31.765 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T04:32:31.821 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:31.822+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:31.887 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:31.888 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:31.902 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T04:32:31.922 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:31.920+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:31.969 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:31.969 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:32:31.986 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-26T04:32:31.330020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":23544352,"num_objects":57,"num_object_clones":6,"num_object_copies":389,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":257,"num_read_kb":7159,"num_write":341,"num_write_kb":27284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":285,"ondisk_log_size":285,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":122,"num_osds":15,"num_per_pool_osds":15,"num_per_pool_omap_osds":15,"kb":1415577600,"kb_used":466112,"kb_used_data":48368,"kb_used_omap":23,"kb_used_meta":417576,"kb_avail":1415111488,"statfs":{"total":1449551462400,"available":1449074163712,"internally_reserved":0,"allocated":49528832,"data_stored":52091927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23926,"internal_metadata":427598474},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,11,2,3,11,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22085632,"num_objects":49,"num_object_clones":6,"num_object_copies":343,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":115,"num_read_kb":7041,"num_write":126,"num_write_kb":22976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.122496"},"pg_stats":[{"pgid":"3.f","version":"35'16","reported_seq":53,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T04:32:28.702250+0000","last_change":"2024-04-26T04:32:23.579102+0000","last_active":"2024-04-26T04:32:28.702250+0000","last_peered":"2024-04-26T04:32:28.702250+0000","last_clean":"2024-04-26T04:32:28.702250+0000","last_became_active":"2024-04-26T04:32:23.378967+0000","last_became_peered":"2024-04-26T04:32:23.378967+0000","last_unstale":"2024-04-26T04:32:28.702250+0000","last_undegraded":"2024-04-26T04:32:28.702250+0000","last_fullsized":"2024-04-26T04:32:28.702250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:38.186716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2375680,"num_objects":6,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":19,"num_read_kb":631,"num_write":16,"num_write_kb":3728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,14,13,8,10,0],"acting":[11,1,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"35'12","reported_seq":44,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T04:32:29.424572+0000","last_change":"2024-04-26T04:32:23.489701+0000","last_active":"2024-04-26T04:32:29.424572+0000","last_peered":"2024-04-26T04:32:29.424572+0000","last_clean":"2024-04-26T04:32:29.424572+0000","last_became_active":"2024-04-26T04:32:23.387938+0000","last_became_peered":"2024-04-26T04:32:23.387938+0000","last_unstale":"2024-04-26T04:32:29.424572+0000","last_undegraded":"2024-04-26T04:32:29.424572+0000","last_fullsized":"2024-04-26T04:32:29.424572+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:51.442401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1523712,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":10,"num_write_kb":1488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,9,1,11,8,6,12],"acting":[13,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.d","version":"35'11","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T04:32:28.598082+0000","last_change":"2024-04-26T04:32:23.594045+0000","last_active":"2024-04-26T04:32:28.598082+0000","last_peered":"2024-04-26T04:32:28.598082+0000","last_clean":"2024-04-26T04:32:28.598082+0000","last_became_active":"2024-04-26T04:32:23.392585+0000","last_became_peered":"2024-04-26T04:32:23.392585+0000","last_unstale":"2024-04-26T04:32:28.598082+0000","last_undegraded":"2024-04-26T04:32:28.598082+0000","last_fullsized":"2024-04-26T04:32:28.598082+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:23.586531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490368,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":965,"num_write":10,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T04:32:28.347972+0000","last_change":"2024-04-26T04:32:23.579580+0000","last_active":"2024-04-26T04:32:28.347972+0000","last_peered":"2024-04-26T04:32:28.347972+0000","last_clean":"2024-04-26T04:32:28.347972+0000","last_became_active":"2024-04-26T04:32:23.577814+0000","last_became_peered":"2024-04-26T04:32:23.577814+0000","last_unstale":"2024-04-26T04:32:28.347972+0000","last_undegraded":"2024-04-26T04:32:28.347972+0000","last_fullsized":"2024-04-26T04:32:28.347972+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:22.791436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"34'9","reported_seq":28,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T04:32:27.382432+0000","last_change":"2024-04-26T04:32:23.221034+0000","last_active":"2024-04-26T04:32:27.382432+0000","last_peered":"2024-04-26T04:32:27.382432+0000","last_clean":"2024-04-26T04:32:27.382432+0000","last_became_active":"2024-04-26T04:32:23.220443+0000","last_became_peered":"2024-04-26T04:32:23.220443+0000","last_unstale":"2024-04-26T04:32:27.382432+0000","last_undegraded":"2024-04-26T04:32:27.382432+0000","last_fullsized":"2024-04-26T04:32:27.382432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:36.363046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2850816,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":9,"num_write_kb":2784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"34'6","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T04:32:28.693737+0000","last_change":"2024-04-26T04:32:23.312351+0000","last_active":"2024-04-26T04:32:28.693737+0000","last_peered":"2024-04-26T04:32:28.693737+0000","last_clean":"2024-04-26T04:32:28.693737+0000","last_became_active":"2024-04-26T04:32:23.310795+0000","last_became_peered":"2024-04-26T04:32:23.310795+0000","last_unstale":"2024-04-26T04:32:28.693737+0000","last_undegraded":"2024-04-26T04:32:28.693737+0000","last_fullsized":"2024-04-26T04:32:28.693737+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:23.941681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966656,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":676,"num_write":6,"num_write_kb":944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"35'4","reported_seq":34,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T04:32:29.368514+0000","last_change":"2024-04-26T04:32:23.477289+0000","last_active":"2024-04-26T04:32:29.368514+0000","last_peered":"2024-04-26T04:32:29.368514+0000","last_clean":"2024-04-26T04:32:29.368514+0000","last_became_active":"2024-04-26T04:32:23.476584+0000","last_became_peered":"2024-04-26T04:32:23.476584+0000","last_unstale":"2024-04-26T04:32:29.368514+0000","last_undegraded":"2024-04-26T04:32:29.368514+0000","last_fullsized":"2024-04-26T04:32:29.368514+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:29.487863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458752,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":1378,"num_write":22,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.8","version":"36'27","reported_seq":65,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T04:32:30.477754+0000","last_change":"2024-04-26T04:32:23.589552+0000","last_active":"2024-04-26T04:32:30.477754+0000","last_peered":"2024-04-26T04:32:30.477754+0000","last_clean":"2024-04-26T04:32:30.477754+0000","last_became_active":"2024-04-26T04:32:23.487565+0000","last_became_peered":"2024-04-26T04:32:23.487565+0000","last_unstale":"2024-04-26T04:32:30.477754+0000","last_undegraded":"2024-04-26T04:32:30.477754+0000","last_fullsized":"2024-04-26T04:32:30.477754+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:25.409254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":10,"num_object_clones":3,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":23,"num_read_kb":3369,"num_write":22,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,12,1,14,15,6],"acting":[5,0,12,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"32'157","reported_seq":242,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T04:32:29.424317+0000","last_change":"2024-04-26T04:32:24.500132+0000","last_active":"2024-04-26T04:32:29.424317+0000","last_peered":"2024-04-26T04:32:29.424317+0000","last_clean":"2024-04-26T04:32:29.424317+0000","last_became_active":"2024-04-26T04:32:24.499360+0000","last_became_peered":"2024-04-26T04:32:24.499360+0000","last_unstale":"2024-04-26T04:32:29.424317+0000","last_undegraded":"2024-04-26T04:32:29.424317+0000","last_fullsized":"2024-04-26T04:32:29.424317+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:32:08.365921+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.833702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3],"acting":[13,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T04:32:29.332395+0000","last_change":"2024-04-26T04:32:12.505010+0000","last_active":"2024-04-26T04:32:29.332395+0000","last_peered":"2024-04-26T04:32:29.332395+0000","last_clean":"2024-04-26T04:32:29.332395+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:32:29.332395+0000","last_undegraded":"2024-04-26T04:32:29.332395+0000","last_fullsized":"2024-04-26T04:32:29.332395+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.374592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"34'7","reported_seq":37,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T04:32:29.524343+0000","last_change":"2024-04-26T04:32:23.397180+0000","last_active":"2024-04-26T04:32:29.524343+0000","last_peered":"2024-04-26T04:32:29.524343+0000","last_clean":"2024-04-26T04:32:29.524343+0000","last_became_active":"2024-04-26T04:32:23.396007+0000","last_became_peered":"2024-04-26T04:32:23.396007+0000","last_unstale":"2024-04-26T04:32:29.524343+0000","last_undegraded":"2024-04-26T04:32:29.524343+0000","last_fullsized":"2024-04-26T04:32:29.524343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.106415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1622016,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":7,"num_write_kb":1584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10,3,6,15,1,11],"acting":[13,10,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T04:32:28.254119+0000","last_change":"2024-04-26T04:32:12.603501+0000","last_active":"2024-04-26T04:32:28.254119+0000","last_peered":"2024-04-26T04:32:28.254119+0000","last_clean":"2024-04-26T04:32:28.254119+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:32:28.254119+0000","last_undegraded":"2024-04-26T04:32:28.254119+0000","last_fullsized":"2024-04-26T04:32:28.254119+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.945207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"33'8","reported_seq":34,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T04:32:26.849269+0000","last_change":"2024-04-26T04:32:23.500550+0000","last_active":"2024-04-26T04:32:26.849269+0000","last_peered":"2024-04-26T04:32:26.849269+0000","last_clean":"2024-04-26T04:32:26.849269+0000","last_became_active":"2024-04-26T04:32:23.399115+0000","last_became_peered":"2024-04-26T04:32:23.399115+0000","last_unstale":"2024-04-26T04:32:26.849269+0000","last_undegraded":"2024-04-26T04:32:26.849269+0000","last_fullsized":"2024-04-26T04:32:26.849269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:46.107728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1130496,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":8,"num_write_kb":1104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,12,13],"acting":[4,11,9,5,10,12,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T04:32:29.424350+0000","last_change":"2024-04-26T04:32:12.505739+0000","last_active":"2024-04-26T04:32:29.424350+0000","last_peered":"2024-04-26T04:32:29.424350+0000","last_clean":"2024-04-26T04:32:29.424350+0000","last_became_active":"2024-04-26T04:32:12.504908+0000","last_became_peered":"2024-04-26T04:32:12.504908+0000","last_unstale":"2024-04-26T04:32:29.424350+0000","last_undegraded":"2024-04-26T04:32:29.424350+0000","last_fullsized":"2024-04-26T04:32:29.424350+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:41.080750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"32'2","reported_seq":31,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T04:32:29.466905+0000","last_change":"2024-04-26T04:32:23.488065+0000","last_active":"2024-04-26T04:32:29.466905+0000","last_peered":"2024-04-26T04:32:29.466905+0000","last_clean":"2024-04-26T04:32:29.466905+0000","last_became_active":"2024-04-26T04:32:23.386975+0000","last_became_peered":"2024-04-26T04:32:23.386975+0000","last_unstale":"2024-04-26T04:32:29.466905+0000","last_undegraded":"2024-04-26T04:32:29.466905+0000","last_fullsized":"2024-04-26T04:32:29.466905+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:36.356252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,11,15,1,5,6,4],"acting":[14,11,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":28,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-04-26T04:32:28.254409+0000","last_change":"2024-04-26T04:32:12.798464+0000","last_active":"2024-04-26T04:32:28.254409+0000","last_peered":"2024-04-26T04:32:28.254409+0000","last_clean":"2024-04-26T04:32:28.254409+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:32:28.254409+0000","last_undegraded":"2024-04-26T04:32:28.254409+0000","last_fullsized":"2024-04-26T04:32:28.254409+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.684262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T04:32:29.524396+0000","last_change":"2024-04-26T04:32:23.489551+0000","last_active":"2024-04-26T04:32:29.524396+0000","last_peered":"2024-04-26T04:32:29.524396+0000","last_clean":"2024-04-26T04:32:29.524396+0000","last_became_active":"2024-04-26T04:32:23.387136+0000","last_became_peered":"2024-04-26T04:32:23.387136+0000","last_unstale":"2024-04-26T04:32:29.524396+0000","last_undegraded":"2024-04-26T04:32:29.524396+0000","last_fullsized":"2024-04-26T04:32:29.524396+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:30.566787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,6,5,3],"acting":[13,7,4,15,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.5","version":"34'10","reported_seq":36,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T04:32:27.241937+0000","last_change":"2024-04-26T04:32:23.296974+0000","last_active":"2024-04-26T04:32:27.241937+0000","last_peered":"2024-04-26T04:32:27.241937+0000","last_clean":"2024-04-26T04:32:27.241937+0000","last_became_active":"2024-04-26T04:32:23.295127+0000","last_became_peered":"2024-04-26T04:32:23.295127+0000","last_unstale":"2024-04-26T04:32:27.241937+0000","last_undegraded":"2024-04-26T04:32:27.241937+0000","last_fullsized":"2024-04-26T04:32:27.241937+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:24.067916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":5,"num_object_clones":0,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":4,"num_write":10,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T04:32:26.327690+0000","last_change":"2024-04-26T04:32:13.108823+0000","last_active":"2024-04-26T04:32:26.327690+0000","last_peered":"2024-04-26T04:32:26.327690+0000","last_clean":"2024-04-26T04:32:26.327690+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:32:26.327690+0000","last_undegraded":"2024-04-26T04:32:26.327690+0000","last_fullsized":"2024-04-26T04:32:26.327690+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"34'8","reported_seq":33,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T04:32:27.242309+0000","last_change":"2024-04-26T04:32:23.391413+0000","last_active":"2024-04-26T04:32:27.242309+0000","last_peered":"2024-04-26T04:32:27.242309+0000","last_clean":"2024-04-26T04:32:27.242309+0000","last_became_active":"2024-04-26T04:32:23.290645+0000","last_became_peered":"2024-04-26T04:32:23.290645+0000","last_unstale":"2024-04-26T04:32:27.242309+0000","last_undegraded":"2024-04-26T04:32:27.242309+0000","last_fullsized":"2024-04-26T04:32:27.242309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:28.057704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1228800,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":7,"num_read_kb":3,"num_write":8,"num_write_kb":1200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,4,1,0,14,13],"acting":[8,9,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T04:32:29.266140+0000","last_change":"2024-04-26T04:32:12.504400+0000","last_active":"2024-04-26T04:32:29.266140+0000","last_peered":"2024-04-26T04:32:29.266140+0000","last_clean":"2024-04-26T04:32:29.266140+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:32:29.266140+0000","last_undegraded":"2024-04-26T04:32:29.266140+0000","last_fullsized":"2024-04-26T04:32:29.266140+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.381954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"34'6","reported_seq":30,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T04:32:27.268658+0000","last_change":"2024-04-26T04:32:23.580018+0000","last_active":"2024-04-26T04:32:27.268658+0000","last_peered":"2024-04-26T04:32:27.268658+0000","last_clean":"2024-04-26T04:32:27.268658+0000","last_became_active":"2024-04-26T04:32:23.477983+0000","last_became_peered":"2024-04-26T04:32:23.477983+0000","last_unstale":"2024-04-26T04:32:27.268658+0000","last_undegraded":"2024-04-26T04:32:27.268658+0000","last_fullsized":"2024-04-26T04:32:27.268658+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:32.240135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1605632,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":6,"num_write_kb":1568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T04:32:26.319461+0000","last_change":"2024-04-26T04:32:12.697838+0000","last_active":"2024-04-26T04:32:26.319461+0000","last_peered":"2024-04-26T04:32:26.319461+0000","last_clean":"2024-04-26T04:32:26.319461+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:32:26.319461+0000","last_undegraded":"2024-04-26T04:32:26.319461+0000","last_fullsized":"2024-04-26T04:32:26.319461+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.961816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"33'2","reported_seq":25,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T04:32:26.732518+0000","last_change":"2024-04-26T04:32:23.485830+0000","last_active":"2024-04-26T04:32:26.732518+0000","last_peered":"2024-04-26T04:32:26.732518+0000","last_clean":"2024-04-26T04:32:26.732518+0000","last_became_active":"2024-04-26T04:32:23.383105+0000","last_became_peered":"2024-04-26T04:32:23.383105+0000","last_unstale":"2024-04-26T04:32:26.732518+0000","last_undegraded":"2024-04-26T04:32:26.732518+0000","last_fullsized":"2024-04-26T04:32:26.732518+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:38.244864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540672,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":2,"num_write_kb":528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":26,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T04:32:26.348378+0000","last_change":"2024-04-26T04:32:12.803651+0000","last_active":"2024-04-26T04:32:26.348378+0000","last_peered":"2024-04-26T04:32:26.348378+0000","last_clean":"2024-04-26T04:32:26.348378+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:32:26.348378+0000","last_undegraded":"2024-04-26T04:32:26.348378+0000","last_fullsized":"2024-04-26T04:32:26.348378+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:26.614652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":22953984,"num_objects":55,"num_object_clones":6,"num_object_copies":385,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":121,"num_read_kb":7041,"num_write":138,"num_write_kb":23824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45109248,"data_stored":51630080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":112,"acting":112,"num_store_stats":15},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1536000,"data_stored":1525344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247816,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30312,"kb_used_data":3296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341528,"statfs":{"total":96636764160,"available":96605724672,"internally_reserved":0,"allocated":3375104,"data_stored":3687552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247817,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30324,"kb_used_data":3308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341516,"statfs":{"total":96636764160,"available":96605712384,"internally_reserved":0,"allocated":3387392,"data_stored":3724416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247816,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35572,"kb_used_data":4396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336268,"statfs":{"total":96636764160,"available":96600338432,"internally_reserved":0,"allocated":4501504,"data_stored":5199520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313223,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31448,"kb_used_data":4428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340392,"statfs":{"total":96636764160,"available":96604561408,"internally_reserved":0,"allocated":4534272,"data_stored":4322837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313223,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30088,"kb_used_data":3072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341752,"statfs":{"total":96636764160,"available":96605954048,"internally_reserved":0,"allocated":3145728,"data_stored":2934293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,3,5,6,7,8,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313223,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30140,"kb_used_data":3124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341700,"statfs":{"total":96636764160,"available":96605900800,"internally_reserved":0,"allocated":3198976,"data_stored":2987541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,2,0,8],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313223,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29092,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342748,"statfs":{"total":96636764160,"available":96606973952,"internally_reserved":0,"allocated":2125824,"data_stored":1914389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313223,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28468,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343372,"statfs":{"total":96636764160,"available":96607612928,"internally_reserved":0,"allocated":1486848,"data_stored":1275413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":98784247817,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280519,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29292,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342548,"statfs":{"total":96636764160,"available":96606769152,"internally_reserved":0,"allocated":2326528,"data_stored":2220058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280519,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34004,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337836,"statfs":{"total":96636764160,"available":96601944064,"internally_reserved":0,"allocated":2891776,"data_stored":3239418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247816,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":4788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":4902912,"data_stored":5686400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247817,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30000,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341840,"statfs":{"total":96636764160,"available":96606044160,"internally_reserved":0,"allocated":3055616,"data_stored":3404928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,8,0,1,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247815,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":4300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":4403200,"data_stored":5166208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280519,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34068,"kb_used_data":2888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337772,"statfs":{"total":96636764160,"available":96601878528,"internally_reserved":0,"allocated":2957312,"data_stored":2838010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247817,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30184,"kb_used_data":3160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341656,"statfs":{"total":96636764160,"available":96605855744,"internally_reserved":0,"allocated":3235840,"data_stored":3490944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4141056,"data_stored":4141056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4476928,"data_stored":5726208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1384448,"data_stored":1499136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2752512,"data_stored":2752512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":3203072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3977216,"data_stored":4964352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2547712,"data_stored":2646016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":2805760,"data_stored":2805760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4038656,"data_stored":4878336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":1732608,"data_stored":1732608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":2482176,"data_stored":2961408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":3833856,"data_stored":4526080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":3543040,"data_stored":4468736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":2961408,"data_stored":3522560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":1802240,"data_stored":1802240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:32:31.988 INFO:tasks.thrashosds.thrasher:Setting 3.0 to [13, 7, 3, 0, 15, 4, 8, 10, 6, 1, 2, 11, 9, 14] 2024-04-26T04:32:31.988 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.0', '13', '7', '3', '0', '15', '4', '8', '10', '6', '1', '2', '11', '9', '14'] 2024-04-26T04:32:31.988 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.0 13 7 3 0 15 4 8 10 6 1 2 11 9 14 2024-04-26T04:32:32.017 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:32.017 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:32.022 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:32.022+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:32.031 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T04:32:32.123 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:32.120+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:32.148 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:32.148 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:32.162 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T04:32:32.225 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:32.222+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:32.232 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-04-26T04:32:32.237 INFO:tasks.rados.rados.0.smithi178.stdout:110: done (6 left) 2024-04-26T04:32:32.238 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 9 current snap is 2 2024-04-26T04:32:32.238 INFO:tasks.rados.rados.0.smithi178.stdout:104: finishing rollback tid 0 to smithi17829308-41 2024-04-26T04:32:32.238 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 46 v 18 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-26T04:32:32.238 INFO:tasks.rados.rados.0.smithi178.stdout:97: finishing write tid 4 to smithi17829308-2 2024-04-26T04:32:32.238 INFO:tasks.rados.rados.0.smithi178.stdout:97: oid 2 updating version 0 to 27 2024-04-26T04:32:32.238 INFO:tasks.rados.rados.0.smithi178.stdout:97: oid 2 updating version 27 to 28 2024-04-26T04:32:32.238 INFO:tasks.rados.rados.0.smithi178.stdout:97: oid 2 updating version 28 to 29 2024-04-26T04:32:32.238 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 2 v 29 (ObjNum 58 snap 1 seq_num 58) dirty exists 2024-04-26T04:32:32.238 INFO:tasks.rados.rados.0.smithi178.stdout:97: left oid 2 (ObjNum 58 snap 1 seq_num 58) 2024-04-26T04:32:32.238 INFO:tasks.rados.rados.0.smithi178.stdout:104: finishing rollback tid 1 to smithi17829308-41 2024-04-26T04:32:32.238 INFO:tasks.rados.rados.0.smithi178.stdout:104: finishing rollback tid 2 to smithi17829308-41 2024-04-26T04:32:32.239 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 41 v 16 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-04-26T04:32:32.239 INFO:tasks.rados.rados.0.smithi178.stdout:108: finishing copy_from racing read to smithi17829308-38 2024-04-26T04:32:32.239 INFO:tasks.rados.rados.0.smithi178.stdout:109: finishing write tid 1 to smithi17829308-18 2024-04-26T04:32:32.239 INFO:tasks.rados.rados.0.smithi178.stdout:109: finishing write tid 2 to smithi17829308-18 2024-04-26T04:32:32.239 INFO:tasks.rados.rados.0.smithi178.stdout:108: finishing copy_from to smithi17829308-38 2024-04-26T04:32:32.239 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 38 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-04-26T04:32:32.239 INFO:tasks.rados.rados.0.smithi178.stdout:103: expect (ObjNum 16 snap 0 seq_num 16) 2024-04-26T04:32:32.259 INFO:tasks.rados.rados.0.smithi178.stdout:109: finishing write tid 3 to smithi17829308-18 2024-04-26T04:32:32.259 INFO:tasks.rados.rados.0.smithi178.stdout:109: finishing write tid 4 to smithi17829308-18 2024-04-26T04:32:32.259 INFO:tasks.rados.rados.0.smithi178.stdout:109: oid 18 updating version 0 to 13 2024-04-26T04:32:32.259 INFO:tasks.rados.rados.0.smithi178.stdout:109: oid 18 version 13 is already newer than 12 2024-04-26T04:32:32.259 INFO:tasks.rados.rados.0.smithi178.stdout:109: oid 18 updating version 13 to 14 2024-04-26T04:32:32.259 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 18 v 14 (ObjNum 62 snap 2 seq_num 62) dirty exists 2024-04-26T04:32:32.260 INFO:tasks.rados.rados.0.smithi178.stdout:109: left oid 18 (ObjNum 62 snap 2 seq_num 62) 2024-04-26T04:32:32.260 INFO:tasks.rados.rados.0.smithi178.stdout:111: seq_num 63 ranges {147456=688128,835584=376832} 2024-04-26T04:32:32.273 INFO:tasks.rados.rados.0.smithi178.stdout:111: writing smithi17829308-9 from 147456 to 835584 tid 1 2024-04-26T04:32:32.275 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:32.275 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:32.279 INFO:tasks.rados.rados.0.smithi178.stdout:111: writing smithi17829308-9 from 835584 to 1212416 tid 2 2024-04-26T04:32:32.279 INFO:tasks.rados.rados.0.smithi178.stdout:97: done (6 left) 2024-04-26T04:32:32.279 INFO:tasks.rados.rados.0.smithi178.stdout:103: done (5 left) 2024-04-26T04:32:32.279 INFO:tasks.rados.rados.0.smithi178.stdout:104: done (4 left) 2024-04-26T04:32:32.280 INFO:tasks.rados.rados.0.smithi178.stdout:106: done (3 left) 2024-04-26T04:32:32.280 INFO:tasks.rados.rados.0.smithi178.stdout:108: done (2 left) 2024-04-26T04:32:32.280 INFO:tasks.rados.rados.0.smithi178.stdout:109: done (1 left) 2024-04-26T04:32:32.280 INFO:tasks.rados.rados.0.smithi178.stdout:112: rollback oid 3 current snap is 2 2024-04-26T04:32:32.280 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 3 to 0 2024-04-26T04:32:32.280 INFO:tasks.rados.rados.0.smithi178.stdout:113: read oid 25 snap -1 2024-04-26T04:32:32.280 INFO:tasks.rados.rados.0.smithi178.stdout:113: expect deleted 2024-04-26T04:32:32.280 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 2 current snap is 2 2024-04-26T04:32:32.280 INFO:tasks.rados.rados.0.smithi178.stdout:114: seq_num 64 ranges {1163264=704512,1867776=376832} 2024-04-26T04:32:32.288 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T04:32:32.309 INFO:tasks.rados.rados.0.smithi178.stdout:114: writing smithi17829308-2 from 1163264 to 1867776 tid 1 2024-04-26T04:32:32.314 INFO:tasks.rados.rados.0.smithi178.stdout:114: writing smithi17829308-2 from 1867776 to 2244608 tid 2 2024-04-26T04:32:32.314 INFO:tasks.rados.rados.0.smithi178.stdout:115: snap_create 2024-04-26T04:32:32.326 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:32.323+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:32.408 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:32.409 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:32.423 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T04:32:32.426 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:32.422+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:32.490 INFO:tasks.rados.rados.0.smithi178.stdout:114: finishing write tid 1 to smithi17829308-2 2024-04-26T04:32:32.490 INFO:tasks.rados.rados.0.smithi178.stdout:114: finishing write tid 2 to smithi17829308-2 2024-04-26T04:32:32.527 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:32.522+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:32.539 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:32.539 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:32.554 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T04:32:32.559 INFO:tasks.rados.rados.0.smithi178.stdout:111: finishing write tid 1 to smithi17829308-9 2024-04-26T04:32:32.559 INFO:tasks.rados.rados.0.smithi178.stdout:111: finishing write tid 2 to smithi17829308-9 2024-04-26T04:32:32.628 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:32.624+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:32.659 INFO:tasks.rados.rados.0.smithi178.stdout:111: finishing write tid 3 to smithi17829308-9 2024-04-26T04:32:32.671 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:32.671 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:32.684 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T04:32:32.694 INFO:tasks.rados.rados.0.smithi178.stdout:114: finishing write tid 3 to smithi17829308-2 2024-04-26T04:32:32.729 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:32.727+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:32.796 INFO:tasks.rados.rados.0.smithi178.stdout:112: finishing rollback tid 0 to smithi17829308-3 2024-04-26T04:32:32.796 INFO:tasks.rados.rados.0.smithi178.stdout:112: finishing rollback tid 1 to smithi17829308-3 2024-04-26T04:32:32.796 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 3 v 22 (ObjNum 129 snap 0 seq_num 2233879832) dirty dne 2024-04-26T04:32:32.804 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:32.805 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:32.819 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T04:32:32.866 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:32.864+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:32.896 INFO:tasks.rados.rados.0.smithi178.stdout:111: finishing write tid 4 to smithi17829308-9 2024-04-26T04:32:32.897 INFO:tasks.rados.rados.0.smithi178.stdout:111: oid 9 updating version 0 to 21 2024-04-26T04:32:32.897 INFO:tasks.rados.rados.0.smithi178.stdout:111: oid 9 version 21 is already newer than 20 2024-04-26T04:32:32.897 INFO:tasks.rados.rados.0.smithi178.stdout:111: oid 9 version 21 is already newer than 19 2024-04-26T04:32:32.897 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 9 v 21 (ObjNum 63 snap 2 seq_num 63) dirty exists 2024-04-26T04:32:32.897 INFO:tasks.rados.rados.0.smithi178.stdout:111: left oid 9 (ObjNum 63 snap 2 seq_num 63) 2024-04-26T04:32:32.930 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:32.930+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:32.938 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:32.939 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:32.952 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T04:32:33.038 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:33.036+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:33.097 INFO:tasks.rados.rados.0.smithi178.stdout:114: finishing write tid 4 to smithi17829308-2 2024-04-26T04:32:33.097 INFO:tasks.rados.rados.0.smithi178.stdout:114: oid 2 updating version 0 to 33 2024-04-26T04:32:33.097 INFO:tasks.rados.rados.0.smithi178.stdout:114: oid 2 version 33 is already newer than 31 2024-04-26T04:32:33.097 INFO:tasks.rados.rados.0.smithi178.stdout:114: oid 2 version 33 is already newer than 32 2024-04-26T04:32:33.097 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 2 v 33 (ObjNum 64 snap 2 seq_num 64) dirty exists 2024-04-26T04:32:33.097 INFO:tasks.rados.rados.0.smithi178.stdout:114: left oid 2 (ObjNum 64 snap 2 seq_num 64) 2024-04-26T04:32:33.113 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:33.113 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:33.127 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T04:32:33.132 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:33.131+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:33.233 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:33.231+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:33.291 INFO:teuthology.orchestra.run.smithi012.stderr:set 3.0 pg_upmap_items mapping to [13->7,3->0,15->4,8->10,6->1,2->11,9->14] 2024-04-26T04:32:33.297 INFO:tasks.rados.rados.0.smithi178.stdout:111: done (4 left) 2024-04-26T04:32:33.297 INFO:tasks.rados.rados.0.smithi178.stdout:112: done (3 left) 2024-04-26T04:32:33.297 INFO:tasks.rados.rados.0.smithi178.stdout:113: done (2 left) 2024-04-26T04:32:33.297 INFO:tasks.rados.rados.0.smithi178.stdout:114: done (1 left) 2024-04-26T04:32:33.297 INFO:tasks.rados.rados.0.smithi178.stdout:115: done (0 left) 2024-04-26T04:32:33.297 INFO:tasks.rados.rados.0.smithi178.stdout:116: read oid 48 snap 2 2024-04-26T04:32:33.297 INFO:tasks.rados.rados.0.smithi178.stdout:116: expect (ObjNum 47 snap 0 seq_num 47) 2024-04-26T04:32:33.297 INFO:tasks.rados.rados.0.smithi178.stdout:117: read oid 49 snap 0 2024-04-26T04:32:33.297 INFO:tasks.rados.rados.0.smithi178.stdout:117: expect (ObjNum 48 snap 0 seq_num 48) 2024-04-26T04:32:33.297 INFO:tasks.rados.rados.0.smithi178.stdout:118: delete oid 44 current snap is 3 2024-04-26T04:32:33.298 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:33.298 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:33.313 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T04:32:33.334 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:33.332+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:33.435 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:33.432+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:33.443 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:33.443 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:33.458 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T04:32:33.536 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:33.532+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:33.581 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:33.582 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:33.596 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T04:32:33.600 INFO:tasks.rados.rados.0.smithi178.stdout:118: done (2 left) 2024-04-26T04:32:33.600 INFO:tasks.rados.rados.0.smithi178.stdout:119: rollback oid 23 current snap is 3 2024-04-26T04:32:33.601 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 23 to 0 2024-04-26T04:32:33.601 INFO:tasks.rados.rados.0.smithi178.stdout:120: read oid 30 snap -1 2024-04-26T04:32:33.601 INFO:tasks.rados.rados.0.smithi178.stdout:120: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-26T04:32:33.601 INFO:tasks.rados.rados.0.smithi178.stdout:121: setattr oid 35 current snap is 3 2024-04-26T04:32:33.602 INFO:tasks.rados.rados.0.smithi178.stdout:122: delete oid 25 current snap is 3 2024-04-26T04:32:33.602 INFO:tasks.rados.rados.0.smithi178.stdout:116: expect (ObjNum 47 snap 0 seq_num 47) 2024-04-26T04:32:33.624 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T04:32:33.636 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:33.635+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:33.730 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:33.730 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:33.737 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:33.732+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:33.745 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T04:32:33.783 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:33.784 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:33.807 INFO:tasks.rados.rados.0.smithi178.stdout:116: done (5 left) 2024-04-26T04:32:33.807 INFO:tasks.rados.rados.0.smithi178.stdout:122: done (4 left) 2024-04-26T04:32:33.807 INFO:tasks.rados.rados.0.smithi178.stdout:123: snap_remove snap 2 2024-04-26T04:32:33.826 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:33.826 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:33.838 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:33.834+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:33.851 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:33.852 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:33.877 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:33.877 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:33.901 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:33.902 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:33.913 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:33.913 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:33.928 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T04:32:33.929 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:33.929 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:33.939 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:33.936+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:33.954 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:33.955 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:33.978 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:33.979 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:34.002 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:34.003 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:34.027 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:34.028 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:34.040 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:34.036+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:34.054 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:34.054 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:34.079 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:34.080 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:34.091 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:34.092 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:34.092 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:32:34.092 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:32:34.092 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:34.105 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:34.105 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:34.106 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T04:32:34.130 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:34.131 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:34.143 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:34.140+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:34.157 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:34.158 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:34.181 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:34.182 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:34.231 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:34.231 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.as1 32/30 ECSubWrite(tid=0, reqid=client.4670.0:40, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.217293+0000", 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.0113746070000005, 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.002088992, 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.232 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.217293+0000", 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.217293+0000", 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.217295+0000", 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0660000000000002e-06 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.233 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.234 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.217422+0000", 2024-04-26T04:32:34.234 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012748600000000001 2024-04-26T04:32:34.234 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.234 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.234 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.234 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.217449+0000", 2024-04-26T04:32:34.234 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7373e-05 2024-04-26T04:32:34.234 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.234 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.234 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.234 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.217463+0000", 2024-04-26T04:32:34.234 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3453e-05 2024-04-26T04:32:34.234 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.235 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.237 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.217585+0000", 2024-04-26T04:32:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012219800000000001 2024-04-26T04:32:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.217602+0000", 2024-04-26T04:32:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6443000000000002e-05 2024-04-26T04:32:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.219311+0000", 2024-04-26T04:32:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001709945 2024-04-26T04:32:34.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.219382+0000", 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.0028000000000001e-05 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.as1 32/30 ECSubWrite(tid=1, reqid=client.4670.0:41, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.219962+0000", 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.008705569, 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101380049, 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.219962+0000", 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.219962+0000", 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.240 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.219963+0000", 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1739999999999999e-06 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.219987+0000", 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4304999999999999e-05 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.220012+0000", 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5208e-05 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.220030+0000", 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7388999999999999e-05 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.320220+0000", 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10018998599999999 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.320235+0000", 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5693e-05 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.321132+0000", 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00089678800000000003 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.321342+0000", 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00020950600000000001 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 35/30 ECSubWrite(tid=28, reqid=client.4670.0:194, at_version=0'0, trim_to=0'0, roll_forward_to=35'16))", 2024-04-26T04:32:34.243 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:28.866109+0000", 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.3625578010000003, 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10211991300000001, 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.866109+0000", 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.244 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.866109+0000", 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.866112+0000", 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3489999999999999e-06 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.866141+0000", 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9445999999999999e-05 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.245 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.866196+0000", 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.5102000000000002e-05 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.966349+0000", 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100152886 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.966545+0000", 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019540399999999999 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.966578+0000", 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3422e-05 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.968076+0000", 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0014983329999999999 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.968229+0000", 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015297100000000001 2024-04-26T04:32:34.247 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.bs5 35/30 ECSubWrite(tid=57, reqid=client.4670.0:176, at_version=35'11, trim_to=0'0, roll_forward_to=34'9))", 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:28.966560+0000", 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.2621068449999999, 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0036431129999999999, 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.248 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.966560+0000", 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.966560+0000", 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.966561+0000", 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.0299999999999998e-07 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.966880+0000", 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00031866899999999999 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.966912+0000", 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1989000000000002e-05 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.966934+0000", 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.23e-05 2024-04-26T04:32:34.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.967065+0000", 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013134900000000001 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.967090+0000", 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4765000000000001e-05 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.970161+0000", 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0030714050000000001 2024-04-26T04:32:34.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.970203+0000", 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.1933e-05 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 35/30 ECSubWriteReply(tid=18, last_complete=35'11, committed=1, applied=1))", 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:28.966982+0000", 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.2616848489999999, 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016220850000000001, 2024-04-26T04:32:34.252 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.966982+0000", 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.966982+0000", 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.254 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.254 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.966983+0000", 2024-04-26T04:32:34.254 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.7699999999999992e-07 2024-04-26T04:32:34.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.254 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.254 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.966991+0000", 2024-04-26T04:32:34.254 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.3889999999999999e-06 2024-04-26T04:32:34.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.254 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.254 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.967012+0000", 2024-04-26T04:32:34.254 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1073e-05 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.967035+0000", 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3626e-05 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.967160+0000", 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000124222 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.968604+0000", 2024-04-26T04:32:34.255 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001444798 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.bs5 35/30 ECSubWrite(tid=59, reqid=client.4670.0:176, at_version=0'0, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:28.970664+0000", 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.258002866, 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10140479400000001, 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.256 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.970664+0000", 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.970664+0000", 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.970666+0000", 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.393e-06 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.257 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.970687+0000", 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1559000000000001e-05 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.970712+0000", 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5219e-05 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.970727+0000", 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4932999999999999e-05 2024-04-26T04:32:34.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.070908+0000", 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100180373 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.070919+0000", 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.111e-05 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.072018+0000", 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001099544 2024-04-26T04:32:34.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.072069+0000", 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0662999999999999e-05 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.as1 36/30 ECSubWrite(tid=15, reqid=client.4670.0:263, at_version=36'8, trim_to=0'0, roll_forward_to=34'6))", 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.819031+0000", 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.4096357939999997, 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.003475527, 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.260 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819031+0000", 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819031+0000", 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.261 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819034+0000", 2024-04-26T04:32:34.262 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1840000000000002e-06 2024-04-26T04:32:34.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.262 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.262 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819092+0000", 2024-04-26T04:32:34.262 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8527999999999999e-05 2024-04-26T04:32:34.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.262 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.262 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819134+0000", 2024-04-26T04:32:34.262 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.2311999999999997e-05 2024-04-26T04:32:34.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819195+0000", 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.0491000000000001e-05 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819408+0000", 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00021325700000000001 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819471+0000", 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.2561e-05 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.263 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.822455+0000", 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0029838949999999999 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.822507+0000", 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.2299000000000002e-05 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 36/30 ECSubWrite(tid=29, reqid=client.4670.0:266, at_version=36'17, trim_to=0'0, roll_forward_to=35'16))", 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.819415+0000", 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.4092521490000003, 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20185976, 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819415+0000", 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819415+0000", 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819416+0000", 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.6000000000000004e-07 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819506+0000", 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.0537000000000004e-05 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.266 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819532+0000", 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5709000000000001e-05 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819552+0000", 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0373e-05 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.019889+0000", 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20033689900000001 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.019905+0000", 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5571999999999999e-05 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.021199+0000", 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001294623 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.021275+0000", 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5487000000000004e-05 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.268 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.as1 36/30 ECSubWrite(tid=16, reqid=client.4670.0:264, at_version=36'9, trim_to=0'0, roll_forward_to=34'6))", 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.821715+0000", 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.4069525550000002, 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016180910000000001, 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.821715+0000", 2024-04-26T04:32:34.269 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.821715+0000", 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.821716+0000", 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8390000000000001e-06 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.270 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.821765+0000", 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.9066000000000001e-05 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.821825+0000", 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9960999999999997e-05 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.821854+0000", 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8918999999999999e-05 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.271 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.821991+0000", 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000136666 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.822007+0000", 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5916000000000001e-05 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.823246+0000", 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001238811 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.272 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.823333+0000", 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.6912999999999999e-05 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.as1 36/30 ECSubWrite(tid=17, reqid=client.4670.0:265, at_version=36'10, trim_to=0'0, roll_forward_to=34'6))", 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.823814+0000", 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.4048534220000004, 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101549895, 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.273 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.823814+0000", 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.823814+0000", 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.274 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.823816+0000", 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8199999999999999e-06 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.823858+0000", 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.2904999999999997e-05 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.823895+0000", 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.6301999999999999e-05 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.275 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.823918+0000", 2024-04-26T04:32:34.276 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3376000000000001e-05 2024-04-26T04:32:34.276 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.276 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.276 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.276 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.924128+0000", 2024-04-26T04:32:34.276 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100209592 2024-04-26T04:32:34.276 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.276 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.276 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.276 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.924142+0000", 2024-04-26T04:32:34.276 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4712000000000001e-05 2024-04-26T04:32:34.276 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.276 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.925296+0000", 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0011537769999999999 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.925364+0000", 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7410999999999995e-05 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.277 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/30 ECSubWrite(tid=71, reqid=client.4670.0:253, at_version=36'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T04:32:34.278 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:30.151446+0000", 2024-04-26T04:32:34.278 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.0772214609999997, 2024-04-26T04:32:34.278 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0033781359999999999, 2024-04-26T04:32:34.278 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.278 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.278 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.278 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.278 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.278 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.278 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.151446+0000", 2024-04-26T04:32:34.278 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.278 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.278 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.151446+0000", 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.151447+0000", 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.593e-06 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.152177+0000", 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00073001700000000001 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.279 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.152207+0000", 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9836e-05 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.152230+0000", 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3223000000000001e-05 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.152411+0000", 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000181094 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.280 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.152441+0000", 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9862e-05 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.154766+0000", 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0023244759999999998 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.154824+0000", 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8035000000000003e-05 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.281 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.as1 36/30 ECSubWrite(tid=18, reqid=client.4670.0:265, at_version=0'0, trim_to=0'0, roll_forward_to=36'10))", 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:30.327162+0000", 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 3.9015047350000001, 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20162941100000001, 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.282 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327162+0000", 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327162+0000", 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327165+0000", 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1639999999999999e-06 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.283 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327194+0000", 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9388999999999999e-05 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327237+0000", 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3065000000000002e-05 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.427384+0000", 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100147443 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.284 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.527653+0000", 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100268697 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.527668+0000", 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4449e-05 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.528724+0000", 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0010562869999999999 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.285 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.528792+0000", 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7916999999999994e-05 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/30 ECSubWrite(tid=80, reqid=client.4670.0:253, at_version=0'0, trim_to=0'0, roll_forward_to=36'13))", 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:30.459277+0000", 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 3.7693897879999998, 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016165579999999999, 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.286 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.459277+0000", 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.459277+0000", 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.287 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.459279+0000", 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8059999999999999e-06 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.459301+0000", 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1466000000000001e-05 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.459330+0000", 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9215999999999999e-05 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.288 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.459353+0000", 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3218999999999999e-05 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.459569+0000", 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00021643300000000001 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.459588+0000", 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8496999999999999e-05 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.289 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.460808+0000", 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001220357 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.460894+0000", 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.5563999999999996e-05 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/30 ECSubWrite(tid=85, reqid=client.4670.0:277, at_version=36'14, trim_to=0'0, roll_forward_to=36'13))", 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:30.966511+0000", 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 3.2621563519999999, 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0014179920000000001, 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.966511+0000", 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.291 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.966511+0000", 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.966513+0000", 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.877e-06 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.966547+0000", 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.4362000000000001e-05 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.292 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.966578+0000", 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0732000000000003e-05 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.966596+0000", 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.806e-05 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.966796+0000", 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00020000000000000001 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.293 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.966811+0000", 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.537e-05 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.967824+0000", 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0010128680000000001 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.967929+0000", 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010472299999999999 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.294 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/30 ECSubWrite(tid=86, reqid=client.4670.0:278, at_version=36'15, trim_to=0'0, roll_forward_to=36'13))", 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:30.968973+0000", 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 3.2596942229999999, 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00157599, 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.295 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.968973+0000", 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.968973+0000", 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.968974+0000", 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3769999999999999e-06 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.296 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.969004+0000", 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9462e-05 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.969028+0000", 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4159e-05 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.969042+0000", 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.379e-05 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.969179+0000", 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000136852 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.969192+0000", 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3767e-05 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.970489+0000", 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001296845 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.298 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.970549+0000", 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9738000000000003e-05 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.bs5 36/30 ECSubWrite(tid=87, reqid=client.4670.0:279, at_version=36'16, trim_to=0'0, roll_forward_to=36'13))", 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:30.970869+0000", 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 3.2577980759999998, 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001296476, 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.299 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.970869+0000", 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.970869+0000", 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.970870+0000", 2024-04-26T04:32:34.300 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1200000000000001e-06 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.970896+0000", 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6122000000000002e-05 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.970916+0000", 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9646000000000001e-05 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.301 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.970928+0000", 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1939999999999999e-05 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.971104+0000", 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017588700000000001 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.971121+0000", 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7176999999999998e-05 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.972084+0000", 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00096314899999999995 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.972166+0000", 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.1434999999999995e-05 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.as1 37/30 ECSubWrite(tid=19, reqid=client.4670.0:285, at_version=37'12, trim_to=0'0, roll_forward_to=36'10))", 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:31.256776+0000", 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 2.9718911119999998, 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10362086299999999, 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.256776+0000", 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.256776+0000", 2024-04-26T04:32:34.304 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.256777+0000", 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.9999999999999996e-07 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.257124+0000", 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00034741600000000002 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.257176+0000", 2024-04-26T04:32:34.305 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.1498000000000001e-05 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.257218+0000", 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.2661999999999999e-05 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.358113+0000", 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10089448099999999 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.306 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.358128+0000", 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5443000000000001e-05 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.360293+0000", 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0021648700000000002 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.360397+0000", 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000103693 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.307 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 39/30 ECSubWriteReply(tid=26, last_complete=39'15, committed=1, applied=1))", 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.010614+0000", 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.21805337899999999, 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100258875, 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010614+0000", 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.308 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010614+0000", 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010616+0000", 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.95e-06 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010626+0000", 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0451999999999999e-05 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010647+0000", 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0454000000000001e-05 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.110731+0000", 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10008418099999999 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.110844+0000", 2024-04-26T04:32:34.310 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000113133 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.110873+0000", 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8705e-05 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 39/30 ECSubWriteReply(tid=26, last_complete=39'15, committed=1, applied=1))", 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.010719+0000", 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.21794780399999999, 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.200420977, 2024-04-26T04:32:34.311 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010719+0000", 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010719+0000", 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.312 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010721+0000", 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.198e-06 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010728+0000", 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.2219999999999996e-06 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010745+0000", 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7180000000000002e-05 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.313 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.110846+0000", 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10010114000000001 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.211083+0000", 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100237346 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.211140+0000", 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.6891000000000001e-05 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.314 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 39/30 ECSubWriteReply(tid=26, last_complete=39'15, committed=1, applied=1))", 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.110825+0000", 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.11784164499999999, 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10049512300000001, 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.110825+0000", 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.110825+0000", 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.110827+0000", 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.398e-06 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.316 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.110837+0000", 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6609999999999995e-06 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.110851+0000", 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4559000000000001e-05 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.110861+0000", 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9369999999999999e-06 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.317 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.318 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.211243+0000", 2024-04-26T04:32:34.318 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100381727 2024-04-26T04:32:34.318 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.318 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.318 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.318 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.211321+0000", 2024-04-26T04:32:34.318 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7841000000000002e-05 2024-04-26T04:32:34.318 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.318 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.318 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.318 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.318 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.318 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:34.319 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-26T04:32:34.321 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T04:32:34.321 INFO:tasks.rados.rados.0.smithi178.stdout:123: done (4 left) 2024-04-26T04:32:34.322 INFO:tasks.rados.rados.0.smithi178.stdout:124: setattr oid 33 current snap is 3 2024-04-26T04:32:34.322 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 35 v 29 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-04-26T04:32:34.322 INFO:tasks.rados.rados.0.smithi178.stdout:119: finishing rollback tid 0 to smithi17829308-23 2024-04-26T04:32:34.322 INFO:tasks.rados.rados.0.smithi178.stdout:119: finishing rollback tid 1 to smithi17829308-23 2024-04-26T04:32:34.322 INFO:tasks.rados.rados.0.smithi178.stdout:119: finishing rollback tid 2 to smithi17829308-23 2024-04-26T04:32:34.322 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 23 v 15 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-04-26T04:32:34.322 INFO:tasks.rados.rados.0.smithi178.stdout:117: expect (ObjNum 48 snap 0 seq_num 48) 2024-04-26T04:32:34.322 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:34.319+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:34.370 INFO:tasks.rados.rados.0.smithi178.stdout:120: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-26T04:32:34.392 INFO:tasks.rados.rados.0.smithi178.stdout:117: done (4 left) 2024-04-26T04:32:34.393 INFO:tasks.rados.rados.0.smithi178.stdout:119: done (3 left) 2024-04-26T04:32:34.393 INFO:tasks.rados.rados.0.smithi178.stdout:120: done (2 left) 2024-04-26T04:32:34.393 INFO:tasks.rados.rados.0.smithi178.stdout:121: done (1 left) 2024-04-26T04:32:34.393 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 49 current snap is 3 2024-04-26T04:32:34.393 INFO:tasks.rados.rados.0.smithi178.stdout:125: seq_num 67 ranges {819200=753664,1572864=376832} 2024-04-26T04:32:34.404 INFO:tasks.rados.rados.0.smithi178.stdout:125: writing smithi17829308-49 from 819200 to 1572864 tid 1 2024-04-26T04:32:34.407 INFO:tasks.rados.rados.0.smithi178.stdout:125: writing smithi17829308-49 from 1572864 to 1949696 tid 2 2024-04-26T04:32:34.407 INFO:tasks.rados.rados.0.smithi178.stdout:126: read oid 48 snap 0 2024-04-26T04:32:34.407 INFO:tasks.rados.rados.0.smithi178.stdout:126: expect (ObjNum 47 snap 0 seq_num 47) 2024-04-26T04:32:34.407 INFO:tasks.rados.rados.0.smithi178.stdout:127: rollback oid 27 current snap is 3 2024-04-26T04:32:34.407 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 27 to 0 2024-04-26T04:32:34.407 INFO:tasks.rados.rados.0.smithi178.stdout:128: delete oid 17 current snap is 3 2024-04-26T04:32:34.420 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:34.418+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:34.501 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:34.502 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:32:34.502 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.502 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.as6 40/30 ECSubWrite(tid=26, reqid=client.4670.0:328, at_version=40'18, trim_to=0'0, roll_forward_to=38'16))", 2024-04-26T04:32:34.502 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.413545+0000", 2024-04-26T04:32:34.502 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.088299047000000006, 2024-04-26T04:32:34.502 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.088341962999999996, 2024-04-26T04:32:34.502 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.502 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.502 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:34.502 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.502 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413545+0000", 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413545+0000", 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413545+0000", 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.2300000000000001e-07 2024-04-26T04:32:34.503 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413592+0000", 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.6789000000000002e-05 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413624+0000", 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2218999999999998e-05 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413651+0000", 2024-04-26T04:32:34.504 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6225999999999999e-05 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.as6 40/30 ECSubWrite(tid=27, reqid=client.4670.0:329, at_version=40'19, trim_to=0'0, roll_forward_to=38'16))", 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.415267+0000", 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.086576899999999998, 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.086706916999999994, 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.505 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.415267+0000", 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.415267+0000", 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.415268+0000", 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2640000000000001e-06 2024-04-26T04:32:34.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.415292+0000", 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3951000000000002e-05 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.415311+0000", 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8909000000000001e-05 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.415323+0000", 2024-04-26T04:32:34.507 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1564e-05 2024-04-26T04:32:34.508 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.508 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.508 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.508 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.508 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-04-26T04:32:34.508 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 2 2024-04-26T04:32:34.508 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:34.518 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T04:32:34.522 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:34.520+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:34.622 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:34.618+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:34.688 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:34.688 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:34.688 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:32:34.688 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:32:34.689 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:34.702 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T04:32:34.712 INFO:tasks.rados.rados.0.smithi178.stdout:126: expect (ObjNum 47 snap 0 seq_num 47) 2024-04-26T04:32:34.723 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:34.723+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:34.740 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 33 v 7 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-04-26T04:32:34.817 INFO:tasks.rados.rados.0.smithi178.stdout:127: finishing rollback tid 0 to smithi17829308-27 2024-04-26T04:32:34.823 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:34.823 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:32:34.823 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:32:34.823 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:32:34.823 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.823 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:7 3.fs0 3:f50d0def:::smithi17829308-3:head [append 0~49152,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:34.823 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.200251+0000", 2024-04-26T04:32:34.823 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.6190549129999994, 2024-04-26T04:32:34.823 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.244505308, 2024-04-26T04:32:34.824 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.824 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.824 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:34.824 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:34.824 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:34.824 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:34.824 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 7 2024-04-26T04:32:34.824 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.824 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.824 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.824 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.825 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200251+0000", 2024-04-26T04:32:34.825 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.825 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.825 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.825 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200251+0000", 2024-04-26T04:32:34.825 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.825 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.825 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.825 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200253+0000", 2024-04-26T04:32:34.825 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8780000000000001e-06 2024-04-26T04:32:34.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.826 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.826 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.826 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200330+0000", 2024-04-26T04:32:34.826 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7019000000000002e-05 2024-04-26T04:32:34.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.826 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.826 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.826 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200350+0000", 2024-04-26T04:32:34.826 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0051000000000002e-05 2024-04-26T04:32:34.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.827 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.827 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200367+0000", 2024-04-26T04:32:34.827 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7323999999999999e-05 2024-04-26T04:32:34.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.827 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.827 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200537+0000", 2024-04-26T04:32:34.827 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017037899999999999 2024-04-26T04:32:34.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.827 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:32:34.828 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200566+0000", 2024-04-26T04:32:34.828 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8639000000000001e-05 2024-04-26T04:32:34.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.828 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.828 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.239605+0000", 2024-04-26T04:32:34.828 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.039038689000000001 2024-04-26T04:32:34.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.828 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:34.828 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.239786+0000", 2024-04-26T04:32:34.828 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018123 2024-04-26T04:32:34.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.829 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.829 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.240538+0000", 2024-04-26T04:32:34.829 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000752347 2024-04-26T04:32:34.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.829 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.829 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.343628+0000", 2024-04-26T04:32:34.830 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103089895 2024-04-26T04:32:34.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.830 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.830 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:34.830 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.444641+0000", 2024-04-26T04:32:34.830 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101012774 2024-04-26T04:32:34.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.830 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.830 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.830 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.444756+0000", 2024-04-26T04:32:34.830 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000115083 2024-04-26T04:32:34.831 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.831 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.831 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.831 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:8 3.fs0 3:f50d0def:::smithi17829308-3:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:34.831 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:32:34.831 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.6188878570000007, 2024-04-26T04:32:34.831 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51428121599999999, 2024-04-26T04:32:34.832 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.832 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.832 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:34.832 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:34.832 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:34.832 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:34.832 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 8 2024-04-26T04:32:34.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.832 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.832 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.832 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.832 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:32:34.833 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.833 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.833 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:32:34.833 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.833 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.833 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200419+0000", 2024-04-26T04:32:34.833 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9800000000000002e-07 2024-04-26T04:32:34.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.834 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.834 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200433+0000", 2024-04-26T04:32:34.834 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4513e-05 2024-04-26T04:32:34.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.834 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.834 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200460+0000", 2024-04-26T04:32:34.834 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6438999999999999e-05 2024-04-26T04:32:34.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.835 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.835 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.835 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200477+0000", 2024-04-26T04:32:34.835 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7346e-05 2024-04-26T04:32:34.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.835 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.835 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.835 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200757+0000", 2024-04-26T04:32:34.835 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000279771 2024-04-26T04:32:34.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.835 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.836 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:34.836 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200766+0000", 2024-04-26T04:32:34.836 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.7159999999999992e-06 2024-04-26T04:32:34.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.836 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.836 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.341113+0000", 2024-04-26T04:32:34.836 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14034733799999999 2024-04-26T04:32:34.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.836 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:34.836 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.341282+0000", 2024-04-26T04:32:34.836 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000169197 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.342226+0000", 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00094361300000000005 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.343875+0000", 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016493910000000001 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:34.837 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714472+0000", 2024-04-26T04:32:34.838 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.37059684599999998 2024-04-26T04:32:34.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.838 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.838 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714699+0000", 2024-04-26T04:32:34.838 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022704800000000001 2024-04-26T04:32:34.838 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.838 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.838 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.838 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:9 3.fs0 3:f50d0def:::smithi17829308-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:34.839 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:32:34.839 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.618807168, 2024-04-26T04:32:34.839 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71712659899999998, 2024-04-26T04:32:34.839 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.839 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.839 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:34.839 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:34.839 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:34.839 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:34.840 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 9 2024-04-26T04:32:34.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.840 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.840 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.840 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.840 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:32:34.840 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.840 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.840 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.840 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:32:34.840 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.841 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.841 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:32:34.841 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1999999999999998e-07 2024-04-26T04:32:34.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.841 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.841 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200506+0000", 2024-04-26T04:32:34.841 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.1399999999999997e-06 2024-04-26T04:32:34.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.842 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.842 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200520+0000", 2024-04-26T04:32:34.842 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4253000000000001e-05 2024-04-26T04:32:34.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.842 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.842 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.842 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200531+0000", 2024-04-26T04:32:34.842 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1375999999999999e-05 2024-04-26T04:32:34.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.842 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.842 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.843 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200817+0000", 2024-04-26T04:32:34.843 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00028585699999999998 2024-04-26T04:32:34.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.843 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.843 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:34.843 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200824+0000", 2024-04-26T04:32:34.843 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6730000000000003e-06 2024-04-26T04:32:34.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.843 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.843 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.843 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.342845+0000", 2024-04-26T04:32:34.844 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.142020961 2024-04-26T04:32:34.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.844 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.844 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:34.844 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.343080+0000", 2024-04-26T04:32:34.844 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000235481 2024-04-26T04:32:34.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.844 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.844 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.844 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714851+0000", 2024-04-26T04:32:34.844 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.371770347 2024-04-26T04:32:34.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.845 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.845 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:34.845 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.715101+0000", 2024-04-26T04:32:34.845 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00025083100000000002 2024-04-26T04:32:34.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.845 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.845 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.845 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917625+0000", 2024-04-26T04:32:34.845 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20252376 2024-04-26T04:32:34.845 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.845 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.846 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.846 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.846 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.846 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:16 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~98304,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:34.846 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:32:34.846 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.6176501670000007, 2024-04-26T04:32:34.846 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.514661445, 2024-04-26T04:32:34.846 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.846 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.846 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:34.847 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:34.847 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:34.847 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:34.847 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 16 2024-04-26T04:32:34.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.849 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:32:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:32:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201657+0000", 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.052e-06 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201822+0000", 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016550000000000001 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201842+0000", 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0231e-05 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.851 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201856+0000", 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3234999999999999e-05 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201960+0000", 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0001043 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201982+0000", 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1936000000000001e-05 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.852 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344166+0000", 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14218376399999999 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344545+0000", 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00037956399999999999 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.346210+0000", 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00166501 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.853 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349256+0000", 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0030461080000000001 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716126+0000", 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.366869679 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716317+0000", 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019106599999999999 2024-04-26T04:32:34.854 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:17 3.fs0 3:f03878c8:::smithi17829308-6:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.6174220760000004, 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51482183299999995, 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:34.855 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 17 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.856 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201885+0000", 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1060000000000001e-06 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201891+0000", 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.806e-06 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201905+0000", 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.432e-05 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.857 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.858 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201929+0000", 2024-04-26T04:32:34.858 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4417e-05 2024-04-26T04:32:34.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.858 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.858 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.858 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202056+0000", 2024-04-26T04:32:34.858 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012684400000000001 2024-04-26T04:32:34.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.858 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.858 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:34.858 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202066+0000", 2024-04-26T04:32:34.858 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.5230000000000001e-06 2024-04-26T04:32:34.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.347195+0000", 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.145129646 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.347408+0000", 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00021322299999999999 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.348249+0000", 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000840363 2024-04-26T04:32:34.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.353641+0000", 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0053921280000000004 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716520+0000", 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.36287924799999999 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716705+0000", 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018520900000000001 2024-04-26T04:32:34.860 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:18 3.fs0 3:f03878c8:::smithi17829308-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.6173589479999997, 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71589294699999995, 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:34.861 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 18 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.862 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201948+0000", 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.11e-06 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201962+0000", 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4297999999999999e-05 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201990+0000", 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8243e-05 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.863 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.864 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.864 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202004+0000", 2024-04-26T04:32:34.864 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3259e-05 2024-04-26T04:32:34.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.864 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.864 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.864 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202132+0000", 2024-04-26T04:32:34.864 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000128596 2024-04-26T04:32:34.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.864 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.864 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:34.865 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202149+0000", 2024-04-26T04:32:34.865 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6463000000000001e-05 2024-04-26T04:32:34.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.865 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.865 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.865 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349005+0000", 2024-04-26T04:32:34.865 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14685641099999999 2024-04-26T04:32:34.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.865 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.866 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:34.866 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349233+0000", 2024-04-26T04:32:34.866 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000227798 2024-04-26T04:32:34.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.866 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.866 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.866 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716805+0000", 2024-04-26T04:32:34.866 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.36757169099999998 2024-04-26T04:32:34.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.866 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.866 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:34.866 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716944+0000", 2024-04-26T04:32:34.866 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013965200000000001 2024-04-26T04:32:34.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.867 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.867 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.867 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917840+0000", 2024-04-26T04:32:34.867 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20089542599999999 2024-04-26T04:32:34.867 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.867 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.867 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.867 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.867 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:37 3.fs0 3:fa2b8868:::smithi17829308-13:head [append 0~212992,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:34.867 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.308319+0000", 2024-04-26T04:32:34.867 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.5109865950000003, 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40929669000000002, 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 37 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308319+0000", 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308319+0000", 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308321+0000", 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.39e-06 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308655+0000", 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00033381099999999999 2024-04-26T04:32:34.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308682+0000", 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7112999999999999e-05 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308703+0000", 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0908999999999999e-05 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349359+0000", 2024-04-26T04:32:34.870 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.040655934999999997 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349607+0000", 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00024798700000000002 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.350900+0000", 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0012927940000000001 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.353649+0000", 2024-04-26T04:32:34.871 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0027486519999999999 2024-04-26T04:32:34.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.872 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.872 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:34.872 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.717506+0000", 2024-04-26T04:32:34.872 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.36385687700000002 2024-04-26T04:32:34.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.872 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.872 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.872 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.717616+0000", 2024-04-26T04:32:34.872 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000110222 2024-04-26T04:32:34.872 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.872 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.872 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:38 3.fs0 3:fa2b8868:::smithi17829308-13:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.308739+0000", 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.5105670769999993, 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50622445199999999, 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 38 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.873 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308739+0000", 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308739+0000", 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308740+0000", 2024-04-26T04:32:34.874 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7570000000000001e-06 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308749+0000", 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.833e-06 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308772+0000", 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2779e-05 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308787+0000", 2024-04-26T04:32:34.875 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5532999999999998e-05 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.351662+0000", 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.042874774999999997 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.351936+0000", 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00027371399999999999 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.876 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.353132+0000", 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0011959539999999999 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.354065+0000", 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000933343 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.814887+0000", 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.46082132199999998 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.877 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.814963+0000", 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.6441999999999998e-05 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.es3 35/30 ECSubWrite(tid=189, reqid=client.4670.0:235, at_version=35'12, trim_to=0'0, roll_forward_to=34'8))", 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:28.917306+0000", 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.9019992380000001, 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201724229, 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.878 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.917306+0000", 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.917306+0000", 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.917308+0000", 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.66e-06 2024-04-26T04:32:34.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.917341+0000", 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2892999999999998e-05 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.917366+0000", 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4752e-05 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:28.917381+0000", 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5227e-05 2024-04-26T04:32:34.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.117689+0000", 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20030789600000001 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.117708+0000", 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8972999999999999e-05 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.118958+0000", 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001250127 2024-04-26T04:32:34.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.119031+0000", 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.2700999999999994e-05 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:241 3.fs0 3:fa2b8868:::smithi17829308-13:2 [read 0~212992,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.767072+0000", 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.0522341129999999, 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20465845899999999, 2024-04-26T04:32:34.882 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 241 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767072+0000", 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.883 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767072+0000", 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767075+0000", 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.5599999999999998e-06 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767089+0000", 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.403e-05 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.884 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767135+0000", 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5361000000000002e-05 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767158+0000", 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3402999999999999e-05 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767321+0000", 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016312100000000001 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767604+0000", 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000282501 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.971730+0000", 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.204126483 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.886 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:242 3.fs0 3:fa2b8868:::smithi17829308-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.767204+0000", 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.0521013870000004, 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.205259003, 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 242 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.887 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767204+0000", 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767204+0000", 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767206+0000", 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6029999999999999e-06 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.888 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767217+0000", 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1212999999999999e-05 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767237+0000", 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0003000000000001e-05 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767247+0000", 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0246e-05 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.889 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.768592+0000", 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001345111 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.768754+0000", 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000161051 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.972463+0000", 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20370977600000001 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.890 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:243 3.fs0 3:fa2b8868:::smithi17829308-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.767317+0000", 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.0519889280000001, 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.307000616, 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 243 2024-04-26T04:32:34.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767317+0000", 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767317+0000", 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767319+0000", 2024-04-26T04:32:34.892 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.81e-06 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767326+0000", 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5499999999999997e-06 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767369+0000", 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3291999999999998e-05 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.893 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767379+0000", 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.2560000000000008e-06 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.869882+0000", 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10250316 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.870039+0000", 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015685499999999999 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.894 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.074317+0000", 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20427869300000001 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpReadReply(3.fs0 36/30 ECSubReadReply(tid=33, attrs_read=0))", 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.769709+0000", 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.0495965529999998, 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20127798599999999, 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.895 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769709+0000", 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769709+0000", 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769709+0000", 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.896 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769709+0000", 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769740+0000", 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0661999999999999e-05 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769756+0000", 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6079999999999999e-05 2024-04-26T04:32:34.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.970918+0000", 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20116167900000001 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.970987+0000", 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9565000000000002e-05 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.898 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpReadReply(3.fs0 36/30 ECSubReadReply(tid=33, attrs_read=0))", 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.769813+0000", 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.0494923260000002, 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201262519, 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769813+0000", 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.899 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769813+0000", 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769814+0000", 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0359999999999999e-06 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769857+0000", 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3099000000000002e-05 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.900 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769878+0000", 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0129e-05 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769888+0000", 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.047e-05 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.971027+0000", 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201138503 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.901 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.971076+0000", 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.9282e-05 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpReadReply(3.fs0 36/30 ECSubReadReply(tid=32, attrs_read=0))", 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.769945+0000", 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.0493608869999997, 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20127229499999999, 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.902 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769945+0000", 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769945+0000", 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.903 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.769945+0000", 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7299999999999995e-07 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.770045+0000", 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9469999999999995e-05 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.770062+0000", 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7263000000000001e-05 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.904 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.770071+0000", 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.6999999999999997e-06 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.971149+0000", 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201077858 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.971217+0000", 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.8330999999999993e-05 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.905 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpReadReply(3.fs0 36/30 ECSubReadReply(tid=34, attrs_read=0))", 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.870830+0000", 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.9484760169999999, 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20232755599999999, 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.870830+0000", 2024-04-26T04:32:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.870830+0000", 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.870831+0000", 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1969999999999999e-06 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.870890+0000", 2024-04-26T04:32:34.907 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9511e-05 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.870910+0000", 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9423e-05 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.870925+0000", 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5325999999999999e-05 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.073048+0000", 2024-04-26T04:32:34.908 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.202122412 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.073157+0000", 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000109687 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpReadReply(3.fs0 36/30 ECSubReadReply(tid=34, attrs_read=0))", 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.871239+0000", 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.9480670529999999, 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20209827899999999, 2024-04-26T04:32:34.909 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.871239+0000", 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.871239+0000", 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.910 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.871240+0000", 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.17e-06 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.871283+0000", 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3084999999999998e-05 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.871300+0000", 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7600000000000001e-05 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.871314+0000", 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3309000000000001e-05 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.073242+0000", 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201927778 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.073337+0000", 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.5336999999999999e-05 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.912 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:281 3.fs0 3:fcc81d86:::smithi17829308-46:head [create,setxattr YOk\\Qsqnjk`Uh[icZasWhcZnad[h[k[Qmk]TihbR (1823) in=40b,setxattr QnfshqY (34) in=7b,setxattr V_tZOrZWhpPekPNs_toQaraRlW\\sWffpapfPROXb (1614) in=40b,setxattr aiTbV]hXcpXjkS]VQonZZkfogbUSnO[bajbZWdRf (839) in=40b,setxattr pZdlecZrV]qp]`sTkf_bSjZVZjhbScXfb[Sr_bsa (18524) in=40b,setxattr kXtOdjofgYOa_Tck\\nYsbVOfgtldphnQ`]P[j]lX (1295) in=40b,setxattr dRpscRhc^OtlbThWSmXVSNge]s (34) in=26b,setxattr T\\s[QdNhlRZh\\\\[X]NpSccmfRcqRaXScZOZ\\ZR^e (1999) in=40b,setxattr YTcfnqlbrRa^rhctrncU_Wa]laamPmYsaqg_f^N` (1471) in=40b,setxattr l_Qeh\\RcafrRYN_YZf_iiaWagZfe[sop]^\\qTggq (561) in=40b,setxattr NShVOQe\\dRYY[tPettNbkgkTVonNshnbYeePWcla (314) in=40b,setxattr ^iVidVQX]Yn`XTY\\pOXm`_PNX`qeckhbQoR`aQWZ (8841) in=40b,setxattr _jkmO]cXrN_S^act\\QOOjpUZOj_Qf^otnZbpWZPU (981) in=40b,setxattr bNbhPX[YP[lppmc`PkWcpnS`XsQNpW\\UYnWr\\_RX (1927) in=40b,setxattr talTebhpYS[[TXcgodoQR\\tf`fVekppjlkWnTqf[ (1753) in=40b,setxattr rp\\XTgd\\X^qaodkhY[elq^RagRhN_NR^j^NmjOUh (806) in=40b,setxattr ]\\oXSo`PWafUO^dbhR^i`hifQgWjRhR[rRa^bnej (70) in=40b,setxattr UkYrdYaZakpnReakOdc\\^bO[YPqfdkdtbNkghaSQ (279) in=40b,setxattr YtUs_hcRdQqgZV]NaadOflcb\\_c^\\Zg_Y\\]snXXZ (701) in=40b,setxattr NRZdh]WclahTe^_btdqe\\^mVYqQOaXaa\\aPVXWnk (19016) in=40b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:31.073634+0000", 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 3.7456715570000001, 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20679287099999999, 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:34.913 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 281 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.073634+0000", 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.073634+0000", 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.914 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.073637+0000", 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.813e-06 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.073734+0000", 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6811000000000005e-05 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.073767+0000", 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2789000000000001e-05 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.915 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.073782+0000", 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5563e-05 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.073918+0000", 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013629199999999999 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.074119+0000", 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00020103100000000001 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.916 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.075811+0000", 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016911109999999999 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.078533+0000", 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0027226529999999998 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.280279+0000", 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201746286 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.280427+0000", 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000147522 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.as6 37/30 ECSubWrite(tid=19, reqid=client.4670.0:285, at_version=37'12, trim_to=0'0, roll_forward_to=36'10))", 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:31.257398+0000", 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 3.561907545, 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.204140929, 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.918 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.257398+0000", 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.257398+0000", 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.257400+0000", 2024-04-26T04:32:34.919 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.626e-06 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.257728+0000", 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00032861000000000001 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.258249+0000", 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00052104099999999997 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.920 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.258281+0000", 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1402000000000002e-05 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.358565+0000", 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10028427400000001 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.358591+0000", 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5664999999999999e-05 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.921 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.461469+0000", 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10287874700000001 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:31.461539+0000", 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9564e-05 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.as6 40/30 ECSubWrite(tid=26, reqid=client.4670.0:328, at_version=40'18, trim_to=0'0, roll_forward_to=38'16))", 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.413545+0000", 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.40576099999999998, 2024-04-26T04:32:34.922 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.202666333, 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413545+0000", 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413545+0000", 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413545+0000", 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.2300000000000001e-07 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413592+0000", 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.6789000000000002e-05 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413624+0000", 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2218999999999998e-05 2024-04-26T04:32:34.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413651+0000", 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6225999999999999e-05 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.513986+0000", 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100335732 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.514017+0000", 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0406e-05 2024-04-26T04:32:34.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616166+0000", 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10214910100000001 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616211+0000", 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5237000000000001e-05 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:34.926 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:34.928 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T04:32:34.929 INFO:tasks.rados.rados.0.smithi178.stdout:125: finishing write tid 1 to smithi17829308-49 2024-04-26T04:32:34.929 INFO:tasks.rados.rados.0.smithi178.stdout:125: finishing write tid 2 to smithi17829308-49 2024-04-26T04:32:34.929 INFO:tasks.rados.rados.0.smithi178.stdout:124: done (4 left) 2024-04-26T04:32:34.929 INFO:tasks.rados.rados.0.smithi178.stdout:126: done (3 left) 2024-04-26T04:32:34.929 INFO:tasks.rados.rados.0.smithi178.stdout:128: done (2 left) 2024-04-26T04:32:34.929 INFO:tasks.rados.rados.0.smithi178.stdout:129: delete oid 43 current snap is 3 2024-04-26T04:32:34.929 INFO:tasks.rados.rados.0.smithi178.stdout:125: finishing write tid 3 to smithi17829308-49 2024-04-26T04:32:34.930 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:34.924+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:35.018 INFO:tasks.rados.rados.0.smithi178.stdout:127: finishing rollback tid 1 to smithi17829308-27 2024-04-26T04:32:35.018 INFO:tasks.rados.rados.0.smithi178.stdout:127: finishing rollback tid 2 to smithi17829308-27 2024-04-26T04:32:35.018 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 27 v 20 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-04-26T04:32:35.029 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:35.026+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:35.099 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:35.099 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:35.099 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 0 2024-04-26T04:32:35.099 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:35.113 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T04:32:35.119 INFO:tasks.rados.rados.0.smithi178.stdout:127: done (2 left) 2024-04-26T04:32:35.119 INFO:tasks.rados.rados.0.smithi178.stdout:129: done (1 left) 2024-04-26T04:32:35.119 INFO:tasks.rados.rados.0.smithi178.stdout:130: copy_from oid 27 from oid 38 current snap is 3 2024-04-26T04:32:35.119 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 31 current snap is 3 2024-04-26T04:32:35.119 INFO:tasks.rados.rados.0.smithi178.stdout:131: seq_num 70 ranges {507904=802816,1310720=376832} 2024-04-26T04:32:35.140 INFO:tasks.rados.rados.0.smithi178.stdout:131: writing smithi17829308-31 from 507904 to 1310720 tid 1 2024-04-26T04:32:35.141 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:35.139+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:35.147 INFO:tasks.rados.rados.0.smithi178.stdout:131: writing smithi17829308-31 from 1310720 to 1687552 tid 2 2024-04-26T04:32:35.148 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 50 current snap is 3 2024-04-26T04:32:35.148 INFO:tasks.rados.rados.0.smithi178.stdout:125: finishing write tid 4 to smithi17829308-49 2024-04-26T04:32:35.148 INFO:tasks.rados.rados.0.smithi178.stdout:125: oid 49 updating version 0 to 20 2024-04-26T04:32:35.148 INFO:tasks.rados.rados.0.smithi178.stdout:125: oid 49 version 20 is already newer than 18 2024-04-26T04:32:35.148 INFO:tasks.rados.rados.0.smithi178.stdout:125: oid 49 version 20 is already newer than 19 2024-04-26T04:32:35.148 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 49 v 20 (ObjNum 67 snap 3 seq_num 67) dirty exists 2024-04-26T04:32:35.148 INFO:tasks.rados.rados.0.smithi178.stdout:125: left oid 49 (ObjNum 67 snap 3 seq_num 67) 2024-04-26T04:32:35.148 INFO:tasks.rados.rados.0.smithi178.stdout:132: seq_num 71 ranges {835584=425984,1261568=573440,1835008=196608} 2024-04-26T04:32:35.167 INFO:tasks.rados.rados.0.smithi178.stdout:132: writing smithi17829308-50 from 835584 to 1261568 tid 1 2024-04-26T04:32:35.178 INFO:tasks.rados.rados.0.smithi178.stdout:132: writing smithi17829308-50 from 1261568 to 1835008 tid 2 2024-04-26T04:32:35.180 INFO:tasks.rados.rados.0.smithi178.stdout:132: writing smithi17829308-50 from 1835008 to 2031616 tid 3 2024-04-26T04:32:35.180 INFO:tasks.rados.rados.0.smithi178.stdout:125: done (3 left) 2024-04-26T04:32:35.181 INFO:tasks.rados.rados.0.smithi178.stdout:133: copy_from oid 41 from oid 16 current snap is 3 2024-04-26T04:32:35.181 INFO:tasks.rados.rados.0.smithi178.stdout:134: copy_from oid 13 from oid 24 current snap is 3 2024-04-26T04:32:35.181 INFO:tasks.rados.rados.0.smithi178.stdout:135: snap_create 2024-04-26T04:32:35.230 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:35.227+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:35.239 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:32:35.239 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:32:35.239 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:35.254 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T04:32:35.311 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-04-26T04:32:35.318 INFO:tasks.rados.rados.0.smithi178.stdout:135: done (5 left) 2024-04-26T04:32:35.318 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 15 current snap is 4 2024-04-26T04:32:35.318 INFO:tasks.rados.rados.0.smithi178.stdout:136: seq_num 72 ranges {245760=442368,688128=720896,1409024=49152} 2024-04-26T04:32:35.323 INFO:tasks.rados.rados.0.smithi178.stdout:136: writing smithi17829308-15 from 245760 to 688128 tid 1 2024-04-26T04:32:35.329 INFO:tasks.rados.rados.0.smithi178.stdout:136: writing smithi17829308-15 from 688128 to 1409024 tid 2 2024-04-26T04:32:35.329 INFO:tasks.rados.rados.0.smithi178.stdout:136: writing smithi17829308-15 from 1409024 to 1458176 tid 3 2024-04-26T04:32:35.329 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 17 current snap is 4 2024-04-26T04:32:35.329 INFO:tasks.rados.rados.0.smithi178.stdout:137: seq_num 73 ranges {0=458752,458752=475136,933888=294912} 2024-04-26T04:32:35.331 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:35.326+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:35.333 INFO:tasks.rados.rados.0.smithi178.stdout:137: writing smithi17829308-17 from 0 to 458752 tid 1 2024-04-26T04:32:35.337 INFO:tasks.rados.rados.0.smithi178.stdout:137: writing smithi17829308-17 from 458752 to 933888 tid 2 2024-04-26T04:32:35.339 INFO:tasks.rados.rados.0.smithi178.stdout:137: writing smithi17829308-17 from 933888 to 1228800 tid 3 2024-04-26T04:32:35.339 INFO:tasks.rados.rados.0.smithi178.stdout:138: rollback oid 26 current snap is 4 2024-04-26T04:32:35.339 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 26 to 0 2024-04-26T04:32:35.339 INFO:tasks.rados.rados.0.smithi178.stdout:139: snap_create 2024-04-26T04:32:35.424 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:35.425 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:32:35.425 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:32:35.425 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:32:35.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.425 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:19 3.5s0 3:a4a2788c:::smithi17829308-7:head [append 0~114688,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:35.425 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.204107+0000", 2024-04-26T04:32:35.425 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 10.217785960000001, 2024-04-26T04:32:35.425 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40419384899999999, 2024-04-26T04:32:35.425 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.425 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.425 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:35.425 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 19 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.204107+0000", 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.204107+0000", 2024-04-26T04:32:35.426 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.204111+0000", 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.6399999999999999e-06 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.204339+0000", 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022846499999999999 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.427 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.204382+0000", 2024-04-26T04:32:35.428 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.2667999999999999e-05 2024-04-26T04:32:35.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.428 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.428 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304483+0000", 2024-04-26T04:32:35.428 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100100909 2024-04-26T04:32:35.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.428 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.428 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.404672+0000", 2024-04-26T04:32:35.428 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100189713 2024-04-26T04:32:35.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.404819+0000", 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014648800000000001 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.405632+0000", 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00081290099999999999 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507692+0000", 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.102059917 2024-04-26T04:32:35.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.430 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:35.430 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608229+0000", 2024-04-26T04:32:35.430 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100537168 2024-04-26T04:32:35.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.430 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.430 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608301+0000", 2024-04-26T04:32:35.430 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1979999999999999e-05 2024-04-26T04:32:35.430 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.430 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.430 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:20 3.5s0 3:a4a2788c:::smithi17829308-7:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304525+0000", 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 10.11736777, 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.41153016100000001, 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 20 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.431 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304525+0000", 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304525+0000", 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304529+0000", 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2559999999999998e-06 2024-04-26T04:32:35.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304561+0000", 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2398999999999999e-05 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304583+0000", 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2079e-05 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304600+0000", 2024-04-26T04:32:35.433 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6626999999999999e-05 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.406379+0000", 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101779781 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.406585+0000", 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000205985 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.407169+0000", 2024-04-26T04:32:35.434 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00058321299999999996 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507899+0000", 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100730483 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.715960+0000", 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20806137199999999 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.435 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716055+0000", 2024-04-26T04:32:35.436 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.4965999999999998e-05 2024-04-26T04:32:35.436 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.436 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.436 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.436 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:21 3.5s0 3:a4a2788c:::smithi17829308-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:35.436 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:32:35.436 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 10.117270039999999, 2024-04-26T04:32:35.436 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71502945500000004, 2024-04-26T04:32:35.436 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.436 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.436 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 21 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.437 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304624+0000", 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.186e-06 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304629+0000", 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3630000000000004e-06 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.438 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.439 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304643+0000", 2024-04-26T04:32:35.439 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4800000000000001e-05 2024-04-26T04:32:35.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.439 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.439 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304654+0000", 2024-04-26T04:32:35.439 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0665999999999999e-05 2024-04-26T04:32:35.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.439 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.439 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507573+0000", 2024-04-26T04:32:35.439 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20291931099999999 2024-04-26T04:32:35.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507644+0000", 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1131999999999996e-05 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.816490+0000", 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30884542700000001 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.816596+0000", 2024-04-26T04:32:35.440 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000105678 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.019652+0000", 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20305689199999999 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:25 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~147456,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 10.117145312, 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61210120899999998, 2024-04-26T04:32:35.441 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 25 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304749+0000", 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.277e-06 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304925+0000", 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017555399999999999 2024-04-26T04:32:35.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304943+0000", 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8529000000000002e-05 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304954+0000", 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0879999999999999e-05 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608582+0000", 2024-04-26T04:32:35.444 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30362831899999998 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608711+0000", 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012887 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609284+0000", 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00057331000000000005 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:35.445 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.710054+0000", 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100769761 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.916732+0000", 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20667817799999999 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.916849+0000", 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000116531 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.446 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:26 3.5s0 3:afb9923e:::smithi17829308-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 10.116923509999999, 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71271566600000003, 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 26 2024-04-26T04:32:35.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.448 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.448 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.448 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.448 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:32:35.448 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.448 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.448 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.448 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:32:35.448 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.448 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.449 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.449 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:32:35.449 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.2399999999999996e-07 2024-04-26T04:32:35.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.449 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.449 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.449 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304975+0000", 2024-04-26T04:32:35.449 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.6460000000000001e-06 2024-04-26T04:32:35.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.449 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.449 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.449 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304984+0000", 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1540000000000008e-06 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304992+0000", 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5259999999999996e-06 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609850+0000", 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.304857769 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.450 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609963+0000", 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011346899999999999 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.610506+0000", 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00054322400000000003 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.710058+0000", 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.099551985999999995 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017545+0000", 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30748642599999998 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017685+0000", 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014054200000000001 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.8s5 32/30 ECSubWrite(tid=3, reqid=client.4670.0:14, at_version=32'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:35.452 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.344008+0000", 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 10.07788513, 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10125525, 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344008+0000", 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.453 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344008+0000", 2024-04-26T04:32:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344009+0000", 2024-04-26T04:32:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1299999999999999e-07 2024-04-26T04:32:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344046+0000", 2024-04-26T04:32:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.6896e-05 2024-04-26T04:32:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344070+0000", 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4295e-05 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344088+0000", 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7713000000000001e-05 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.444299+0000", 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100211811 2024-04-26T04:32:35.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.444313+0000", 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3543999999999999e-05 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.445169+0000", 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00085646400000000003 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.445263+0000", 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.3814000000000007e-05 2024-04-26T04:32:35.456 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/30 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.407840+0000", 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 10.014053257, 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20052521400000001, 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.457 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.407840+0000", 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.407840+0000", 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.407841+0000", 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1739999999999999e-06 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.458 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.459 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.407846+0000", 2024-04-26T04:32:35.459 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.9749999999999998e-06 2024-04-26T04:32:35.459 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.459 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.459 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.459 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.407859+0000", 2024-04-26T04:32:35.459 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3455000000000001e-05 2024-04-26T04:32:35.459 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.459 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.459 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.460 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.407869+0000", 2024-04-26T04:32:35.460 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.3710000000000006e-06 2024-04-26T04:32:35.460 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.460 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.460 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.460 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608352+0000", 2024-04-26T04:32:35.460 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20048358199999999 2024-04-26T04:32:35.460 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.460 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.460 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.460 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608365+0000", 2024-04-26T04:32:35.460 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2656999999999999e-05 2024-04-26T04:32:35.460 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/30 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.408067+0000", 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 10.013826346, 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20033698, 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.461 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.462 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.408067+0000", 2024-04-26T04:32:35.462 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.462 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.462 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.462 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.408067+0000", 2024-04-26T04:32:35.462 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.462 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.462 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.462 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.408067+0000", 2024-04-26T04:32:35.462 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7499999999999999e-07 2024-04-26T04:32:35.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.463 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.463 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.463 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.408071+0000", 2024-04-26T04:32:35.463 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.6200000000000001e-06 2024-04-26T04:32:35.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.463 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.463 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.463 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.408082+0000", 2024-04-26T04:32:35.463 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0916e-05 2024-04-26T04:32:35.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.463 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.463 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.463 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.408099+0000", 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7136e-05 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608393+0000", 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.200294219 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608404+0000", 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0314e-05 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.464 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 32/30 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.414466+0000", 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 10.007426711999999, 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.19362249100000001, 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.414466+0000", 2024-04-26T04:32:35.465 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.466 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.466 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.414466+0000", 2024-04-26T04:32:35.466 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.466 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.466 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.414467+0000", 2024-04-26T04:32:35.466 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0109999999999999e-06 2024-04-26T04:32:35.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.466 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.414473+0000", 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.1739999999999999e-06 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.414489+0000", 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6977e-05 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.414498+0000", 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.0509999999999997e-06 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.467 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608075+0000", 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.19357727199999999 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608089+0000", 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4005999999999999e-05 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 32/30 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-04-26T04:32:35.468 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.417788+0000", 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 10.004104831999999, 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.19069385599999999, 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.417788+0000", 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.469 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.417788+0000", 2024-04-26T04:32:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.417789+0000", 2024-04-26T04:32:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0270000000000001e-06 2024-04-26T04:32:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.417795+0000", 2024-04-26T04:32:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.224e-06 2024-04-26T04:32:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.417814+0000", 2024-04-26T04:32:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8587999999999999e-05 2024-04-26T04:32:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.417826+0000", 2024-04-26T04:32:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1797000000000001e-05 2024-04-26T04:32:35.471 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.473 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.474 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.474 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608472+0000", 2024-04-26T04:32:35.474 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.19064637200000001 2024-04-26T04:32:35.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.474 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.474 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.474 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608482+0000", 2024-04-26T04:32:35.474 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8479999999999996e-06 2024-04-26T04:32:35.474 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.474 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.474 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.474 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/30 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.506935+0000", 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.9149582790000004, 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10109098499999999, 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.506935+0000", 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.475 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.506935+0000", 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.506936+0000", 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1900000000000001e-07 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.506941+0000", 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.5079999999999996e-06 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.506956+0000", 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4565e-05 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.506963+0000", 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7500000000000003e-06 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608005+0000", 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10104173900000001 2024-04-26T04:32:35.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608026+0000", 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0503999999999999e-05 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpRead(3.0s2 33/30 ECSubRead(tid=3, to_read={3:0061323f:::smithi17829308-8:head=0,4096,0}, subchunks={3:0061323f:::smithi17829308-8:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.615502+0000", 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.8063905410000007, 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100792959, 2024-04-26T04:32:35.478 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.615502+0000", 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.615502+0000", 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.479 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.615503+0000", 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.3099999999999998e-07 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.615513+0000", 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9960000000000005e-06 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.615527+0000", 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3855e-05 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.480 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.715621+0000", 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100093685 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.715697+0000", 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.6050999999999994e-05 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716295+0000", 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00059884100000000002 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.481 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/30 ECSubWriteReply(tid=2, last_complete=33'3, committed=1, applied=1))", 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.810647+0000", 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.6112463199999993, 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10623705999999999, 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.482 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.810647+0000", 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.810647+0000", 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.810649+0000", 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2479999999999999e-06 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.483 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.810661+0000", 2024-04-26T04:32:35.484 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1681e-05 2024-04-26T04:32:35.484 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.484 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.484 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.484 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.810684+0000", 2024-04-26T04:32:35.484 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3717999999999999e-05 2024-04-26T04:32:35.484 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.484 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.484 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.484 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.810697+0000", 2024-04-26T04:32:35.484 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2335e-05 2024-04-26T04:32:35.484 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.484 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.485 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.485 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.916615+0000", 2024-04-26T04:32:35.485 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10591811800000001 2024-04-26T04:32:35.485 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.485 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.485 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.485 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.916884+0000", 2024-04-26T04:32:35.485 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026896 2024-04-26T04:32:35.485 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.485 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.485 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.485 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.485 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpReadReply(3.5s0 33/30 ECSubReadReply(tid=4, attrs_read=0))", 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.819712+0000", 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.6021812620000002, 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.197478129, 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.819712+0000", 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.486 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.819712+0000", 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.819713+0000", 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.33e-06 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.819737+0000", 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3811999999999998e-05 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.487 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.819786+0000", 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.8698e-05 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.819803+0000", 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.73e-05 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017080+0000", 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.197277549 2024-04-26T04:32:35.488 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017190+0000", 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010944 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpReadReply(3.5s0 33/30 ECSubReadReply(tid=4, attrs_read=0))", 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.911686+0000", 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.5102072080000006, 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.105681995, 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.489 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.911686+0000", 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.911686+0000", 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.490 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.911688+0000", 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6899999999999999e-06 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.911708+0000", 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0392999999999999e-05 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.911727+0000", 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9355e-05 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.491 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.911739+0000", 2024-04-26T04:32:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1343e-05 2024-04-26T04:32:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017276+0000", 2024-04-26T04:32:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.105537747 2024-04-26T04:32:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017368+0000", 2024-04-26T04:32:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1466999999999997e-05 2024-04-26T04:32:35.492 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/30 ECSubWriteReply(tid=3, last_complete=33'4, committed=1, applied=1))", 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.911955+0000", 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.5099377379999996, 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10650245699999999, 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.493 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.911955+0000", 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.911955+0000", 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.911956+0000", 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0900000000000002e-07 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.494 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.911960+0000", 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7610000000000001e-06 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.911970+0000", 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0763e-05 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.012066+0000", 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100095393 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.495 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017421+0000", 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0053557270000000002 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.018458+0000", 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001036304 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpReadReply(3.5s0 33/30 ECSubReadReply(tid=4, attrs_read=0))", 2024-04-26T04:32:35.496 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.917237+0000", 2024-04-26T04:32:35.497 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.5046556619999993, 2024-04-26T04:32:35.497 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.102430311, 2024-04-26T04:32:35.497 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.497 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.497 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.497 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.497 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.497 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.497 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917237+0000", 2024-04-26T04:32:35.497 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.497 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.498 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.498 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.498 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917237+0000", 2024-04-26T04:32:35.498 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.498 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.498 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.498 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.498 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917237+0000", 2024-04-26T04:32:35.498 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.498 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.498 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.498 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.498 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917237+0000", 2024-04-26T04:32:35.499 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.499 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.499 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.499 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.499 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917278+0000", 2024-04-26T04:32:35.499 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.0645999999999997e-05 2024-04-26T04:32:35.499 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.499 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.499 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.499 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917293+0000", 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5122e-05 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.019487+0000", 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.102193481 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.019668+0000", 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018106200000000001 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.500 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.501 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.501 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 33/30 ECSubWriteReply(tid=5, last_complete=33'4, committed=1, applied=1))", 2024-04-26T04:32:35.501 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:26.019906+0000", 2024-04-26T04:32:35.501 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.4019871580000007, 2024-04-26T04:32:35.501 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100800929, 2024-04-26T04:32:35.501 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.501 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.501 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.501 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.501 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.501 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.501 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.019906+0000", 2024-04-26T04:32:35.501 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.019906+0000", 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.019907+0000", 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.257e-06 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.502 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.019915+0000", 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.0760000000000003e-06 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.019940+0000", 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4669e-05 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.019956+0000", 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6277e-05 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.503 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.504 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.120679+0000", 2024-04-26T04:32:35.504 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10072262 2024-04-26T04:32:35.504 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.504 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.504 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.504 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.120707+0000", 2024-04-26T04:32:35.504 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8030000000000001e-05 2024-04-26T04:32:35.504 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.504 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.504 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.504 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.504 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.504 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:266 3.5s0 3:a5af5236:::smithi17829308-45:head [stat,delete] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:35.505 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.817699+0000", 2024-04-26T04:32:35.505 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.6041939559999996, 2024-04-26T04:32:35.505 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30440724800000002, 2024-04-26T04:32:35.505 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.505 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.505 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:35.505 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:35.505 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:35.505 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:35.505 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 266 2024-04-26T04:32:35.505 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.505 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.505 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817699+0000", 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817699+0000", 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817702+0000", 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.4029999999999999e-06 2024-04-26T04:32:35.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817718+0000", 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.558e-05 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817754+0000", 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.5908999999999998e-05 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817778+0000", 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4221999999999999e-05 2024-04-26T04:32:35.507 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817892+0000", 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000114041 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.818024+0000", 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013207099999999999 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.818890+0000", 2024-04-26T04:32:35.508 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00086597099999999999 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.820245+0000", 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001354856 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.121917+0000", 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30167228699999998 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.122106+0000", 2024-04-26T04:32:35.509 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000188908 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 36/30 ECSubWriteReply(tid=29, last_complete=36'17, committed=1, applied=1))", 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:30.021385+0000", 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.4005078170000003, 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101547953, 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:35.510 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.021385+0000", 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.021385+0000", 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.021387+0000", 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1270000000000001e-06 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.511 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.021397+0000", 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8800000000000003e-06 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.021426+0000", 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8799e-05 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.021441+0000", 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5146e-05 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.512 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.121688+0000", 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100246433 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.122933+0000", 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0012455680000000001 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:35.513 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:35.515 INFO:tasks.rados.rados.0.smithi178.stdout:132: finishing write tid 1 to smithi17829308-50 2024-04-26T04:32:35.515 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T04:32:35.516 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:35.510+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:35.543 INFO:tasks.rados.rados.0.smithi178.stdout:131: finishing write tid 1 to smithi17829308-31 2024-04-26T04:32:35.543 INFO:tasks.rados.rados.0.smithi178.stdout:131: finishing write tid 2 to smithi17829308-31 2024-04-26T04:32:35.579 INFO:tasks.rados.rados.0.smithi178.stdout:130: finishing copy_from racing read to smithi17829308-27 2024-04-26T04:32:35.617 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:35.612+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:35.643 INFO:tasks.rados.rados.0.smithi178.stdout:131: finishing write tid 3 to smithi17829308-31 2024-04-26T04:32:35.649 INFO:tasks.rados.rados.0.smithi178.stdout:133: finishing copy_from racing read to smithi17829308-41 2024-04-26T04:32:35.659 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:35.659 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:32:35.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.659 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.5s5 41/30 ECSubWrite(tid=38, reqid=client.4670.0:360, at_version=41'24, trim_to=0'0, roll_forward_to=38'21))", 2024-04-26T04:32:35.659 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.578081+0000", 2024-04-26T04:32:35.659 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.079276133999999998, 2024-04-26T04:32:35.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.079319911000000007, 2024-04-26T04:32:35.659 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.578081+0000", 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.578081+0000", 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.578083+0000", 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3379999999999999e-06 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.578113+0000", 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0516000000000001e-05 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.578132+0000", 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9162000000000001e-05 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.578146+0000", 2024-04-26T04:32:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3797e-05 2024-04-26T04:32:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 41/30 ECSubWriteReply(tid=17, last_complete=40'10, committed=1, applied=1))", 2024-04-26T04:32:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.578424+0000", 2024-04-26T04:32:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.078932884999999994, 2024-04-26T04:32:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.079064353000000004, 2024-04-26T04:32:35.662 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "no flag points reached", 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.578424+0000", 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.578424+0000", 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.578426+0000", 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6199999999999999e-06 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.578437+0000", 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0923e-05 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.578461+0000", 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4324e-05 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.664 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.5s5 41/30 ECSubWrite(tid=39, reqid=client.4670.0:361, at_version=41'25, trim_to=0'0, roll_forward_to=38'21))", 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.580021+0000", 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.077336515999999994, 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.077537537000000004, 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:35.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.580021+0000", 2024-04-26T04:32:35.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:35.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.580021+0000", 2024-04-26T04:32:35.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:35.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.580022+0000", 2024-04-26T04:32:35.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1069999999999999e-06 2024-04-26T04:32:35.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:35.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.580043+0000", 2024-04-26T04:32:35.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0769000000000001e-05 2024-04-26T04:32:35.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:35.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.580060+0000", 2024-04-26T04:32:35.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7082e-05 2024-04-26T04:32:35.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.580074+0000", 2024-04-26T04:32:35.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3811999999999999e-05 2024-04-26T04:32:35.668 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.668 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:35.668 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.668 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:340 3.6s0 3:674cc614:::smithi17829308-50:head [append 0~573440 in=573440b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:35.668 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.177236+0000", 2024-04-26T04:32:35.669 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.48012096300000001, 2024-04-26T04:32:35.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.48041209800000001, 2024-04-26T04:32:35.669 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:35.669 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:35.669 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:35.669 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:35.669 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:35.669 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:35.669 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 340 2024-04-26T04:32:35.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.669 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:35.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:35.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.177236+0000", 2024-04-26T04:32:35.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:35.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.177236+0000", 2024-04-26T04:32:35.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:35.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.177239+0000", 2024-04-26T04:32:35.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5100000000000001e-06 2024-04-26T04:32:35.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:35.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178007+0000", 2024-04-26T04:32:35.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00076857200000000003 2024-04-26T04:32:35.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:35.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178041+0000", 2024-04-26T04:32:35.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3909999999999999e-05 2024-04-26T04:32:35.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178057+0000", 2024-04-26T04:32:35.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5906999999999999e-05 2024-04-26T04:32:35.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:35.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178187+0000", 2024-04-26T04:32:35.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012976099999999999 2024-04-26T04:32:35.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:35.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178351+0000", 2024-04-26T04:32:35.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016396800000000001 2024-04-26T04:32:35.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:35.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.179822+0000", 2024-04-26T04:32:35.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001470507 2024-04-26T04:32:35.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:35.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.185617+0000", 2024-04-26T04:32:35.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0057954089999999996 2024-04-26T04:32:35.673 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.673 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:35.674 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.674 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:341 3.6s0 3:674cc614:::smithi17829308-50:head [append 0~196608 in=196608b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:35.674 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180077+0000", 2024-04-26T04:32:35.674 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.47727989900000001, 2024-04-26T04:32:35.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.47770072899999999, 2024-04-26T04:32:35.674 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:35.674 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:35.674 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:35.675 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:35.675 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:35.675 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:35.675 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 341 2024-04-26T04:32:35.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.675 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:35.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:35.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180077+0000", 2024-04-26T04:32:35.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:35.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180077+0000", 2024-04-26T04:32:35.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:35.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180081+0000", 2024-04-26T04:32:35.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2459999999999999e-06 2024-04-26T04:32:35.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180497+0000", 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00041676000000000001 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180529+0000", 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1497e-05 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180556+0000", 2024-04-26T04:32:35.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6594e-05 2024-04-26T04:32:35.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:35.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.181275+0000", 2024-04-26T04:32:35.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00071977499999999997 2024-04-26T04:32:35.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:35.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.181487+0000", 2024-04-26T04:32:35.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211855 2024-04-26T04:32:35.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.182884+0000", 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00139722 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287222+0000", 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10433723 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:342 3.6s0 3:674cc614:::smithi17829308-50:head [setxattr _header (56) in=63b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:35.679 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180608+0000", 2024-04-26T04:32:35.680 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.47674926499999998, 2024-04-26T04:32:35.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.47729755200000001, 2024-04-26T04:32:35.680 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:35.680 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:35.680 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:35.681 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:35.681 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 342 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180608+0000", 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180608+0000", 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:35.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180610+0000", 2024-04-26T04:32:35.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8029999999999999e-06 2024-04-26T04:32:35.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:35.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180619+0000", 2024-04-26T04:32:35.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3170000000000008e-06 2024-04-26T04:32:35.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180642+0000", 2024-04-26T04:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2442999999999998e-05 2024-04-26T04:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180658+0000", 2024-04-26T04:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6779999999999999e-05 2024-04-26T04:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.183978+0000", 2024-04-26T04:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003319598 2024-04-26T04:32:35.684 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:35.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.184127+0000", 2024-04-26T04:32:35.685 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014915100000000001 2024-04-26T04:32:35.685 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.685 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.685 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:35.685 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.185029+0000", 2024-04-26T04:32:35.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00090211999999999996 2024-04-26T04:32:35.686 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.686 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.686 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:35.686 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287425+0000", 2024-04-26T04:32:35.686 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10239564600000001 2024-04-26T04:32:35.686 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.687 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:35.687 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.687 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.687 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.687 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:343 3.6s0 3:674cc614:::smithi17829308-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:35.687 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180684+0000", 2024-04-26T04:32:35.687 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.47667337799999998, 2024-04-26T04:32:35.687 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.47735053300000002, 2024-04-26T04:32:35.688 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:35.688 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:35.688 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for rw locks", 2024-04-26T04:32:35.688 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:35.688 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:35.688 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:35.688 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 343 2024-04-26T04:32:35.688 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.688 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:35.688 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.688 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:35.688 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180684+0000", 2024-04-26T04:32:35.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.689 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.689 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.689 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:35.689 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180684+0000", 2024-04-26T04:32:35.689 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180685+0000", 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2869999999999999e-06 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180691+0000", 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0929999999999999e-06 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:35.690 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180712+0000", 2024-04-26T04:32:35.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0514e-05 2024-04-26T04:32:35.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:35.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180728+0000", 2024-04-26T04:32:35.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5665999999999999e-05 2024-04-26T04:32:35.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.691 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:35.691 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.286928+0000", 2024-04-26T04:32:35.691 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.106200577 2024-04-26T04:32:35.691 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:35.691 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:35.692 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:35.692 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287076+0000", 2024-04-26T04:32:35.692 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014773100000000001 2024-04-26T04:32:35.692 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.692 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:35.692 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.692 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:35.692 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:32:35.692 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 7 2024-04-26T04:32:35.692 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:35.693 INFO:tasks.rados.rados.0.smithi178.stdout:132: finishing write tid 2 to smithi17829308-50 2024-04-26T04:32:35.693 INFO:tasks.rados.rados.0.smithi178.stdout:132: finishing write tid 3 to smithi17829308-50 2024-04-26T04:32:35.694 INFO:tasks.rados.rados.0.smithi178.stdout:132: finishing write tid 4 to smithi17829308-50 2024-04-26T04:32:35.694 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T04:32:35.717 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:35.716+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:35.783 INFO:tasks.rados.rados.0.smithi178.stdout:130: finishing copy_from to smithi17829308-27 2024-04-26T04:32:35.783 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 27 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-04-26T04:32:35.783 INFO:tasks.rados.rados.0.smithi178.stdout:138: finishing rollback tid 0 to smithi17829308-26 2024-04-26T04:32:35.788 INFO:tasks.rados.rados.0.smithi178.stdout:134: finishing copy_from racing read to smithi17829308-13 2024-04-26T04:32:35.818 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:35.816+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:35.868 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:32:35.868 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-04-26T04:32:35.868 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-04-26T04:32:35.868 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:35.881 INFO:tasks.rados.rados.0.smithi178.stdout:131: finishing write tid 4 to smithi17829308-31 2024-04-26T04:32:35.881 INFO:tasks.rados.rados.0.smithi178.stdout:131: oid 31 updating version 0 to 12 2024-04-26T04:32:35.881 INFO:tasks.rados.rados.0.smithi178.stdout:131: oid 31 updating version 12 to 13 2024-04-26T04:32:35.881 INFO:tasks.rados.rados.0.smithi178.stdout:131: oid 31 version 13 is already newer than 11 2024-04-26T04:32:35.881 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 31 v 13 (ObjNum 70 snap 3 seq_num 70) dirty exists 2024-04-26T04:32:35.881 INFO:tasks.rados.rados.0.smithi178.stdout:131: left oid 31 (ObjNum 70 snap 3 seq_num 70) 2024-04-26T04:32:35.884 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T04:32:35.884 INFO:tasks.rados.rados.0.smithi178.stdout:138: finishing rollback tid 1 to smithi17829308-26 2024-04-26T04:32:35.892 INFO:tasks.rados.rados.0.smithi178.stdout:136: finishing write tid 1 to smithi17829308-15 2024-04-26T04:32:35.892 INFO:tasks.rados.rados.0.smithi178.stdout:136: finishing write tid 2 to smithi17829308-15 2024-04-26T04:32:35.892 INFO:tasks.rados.rados.0.smithi178.stdout:132: finishing write tid 5 to smithi17829308-50 2024-04-26T04:32:35.892 INFO:tasks.rados.rados.0.smithi178.stdout:132: oid 50 updating version 0 to 9 2024-04-26T04:32:35.892 INFO:tasks.rados.rados.0.smithi178.stdout:132: oid 50 updating version 9 to 12 2024-04-26T04:32:35.892 INFO:tasks.rados.rados.0.smithi178.stdout:132: oid 50 version 12 is already newer than 10 2024-04-26T04:32:35.892 INFO:tasks.rados.rados.0.smithi178.stdout:132: oid 50 version 12 is already newer than 11 2024-04-26T04:32:35.892 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 50 v 12 (ObjNum 71 snap 3 seq_num 71) dirty exists 2024-04-26T04:32:35.893 INFO:tasks.rados.rados.0.smithi178.stdout:132: left oid 50 (ObjNum 71 snap 3 seq_num 71) 2024-04-26T04:32:35.919 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:35.919+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:35.950 INFO:tasks.rados.rados.0.smithi178.stdout:137: finishing write tid 1 to smithi17829308-17 2024-04-26T04:32:35.950 INFO:tasks.rados.rados.0.smithi178.stdout:133: finishing copy_from to smithi17829308-41 2024-04-26T04:32:35.950 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 41 v 6 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-04-26T04:32:35.984 INFO:tasks.rados.rados.0.smithi178.stdout:138: finishing rollback tid 2 to smithi17829308-26 2024-04-26T04:32:35.984 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 26 v 25 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-04-26T04:32:35.990 INFO:tasks.rados.rados.0.smithi178.stdout:134: finishing copy_from to smithi17829308-13 2024-04-26T04:32:35.990 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 13 v 14 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-04-26T04:32:36.015 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:221 3.6s0 3:674cc614:::smithi17829308-50:2 [read 0~835584,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:28.557487+0000", 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 7.4557035660000004, 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20270885399999999, 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:36.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 221 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557487+0000", 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557487+0000", 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557489+0000", 2024-04-26T04:32:36.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0159999999999998e-06 2024-04-26T04:32:36.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557497+0000", 2024-04-26T04:32:36.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0139999999999998e-06 2024-04-26T04:32:36.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557516+0000", 2024-04-26T04:32:36.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8987999999999998e-05 2024-04-26T04:32:36.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557526+0000", 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7359999999999996e-06 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557694+0000", 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016748500000000001 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557850+0000", 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015615200000000001 2024-04-26T04:32:36.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.760196+0000", 2024-04-26T04:32:36.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.202346463 2024-04-26T04:32:36.020 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.020 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.020 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:222 3.6s0 3:674cc614:::smithi17829308-50:2 [read 0~835584,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:28.557550+0000", 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 7.4556410480000004, 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20363853700000001, 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 222 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.021 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557550+0000", 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557550+0000", 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557550+0000", 2024-04-26T04:32:36.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2200000000000004e-07 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557555+0000", 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6110000000000004e-06 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557563+0000", 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.605e-06 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557568+0000", 2024-04-26T04:32:36.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9070000000000004e-06 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.558367+0000", 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00079864699999999999 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.558618+0000", 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025066299999999998 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.761188+0000", 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.202570582 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:223 3.6s0 3:674cc614:::smithi17829308-50:2 [read 0~835584,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:28.557631+0000", 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 7.4555596680000003, 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.30457677100000002, 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:36.025 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 223 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557631+0000", 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557631+0000", 2024-04-26T04:32:36.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557632+0000", 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0800000000000004e-07 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557636+0000", 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8489999999999999e-06 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557653+0000", 2024-04-26T04:32:36.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6906e-05 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.557670+0000", 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7195000000000001e-05 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.758905+0000", 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.201235461 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.759073+0000", 2024-04-26T04:32:36.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016749700000000001 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.862208+0000", 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.103135255 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(osd.4.21:4 3.6s0 3:674cc614:::smithi17829308-50:head [assert-version v5,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:30.049630+0000", 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.9635610479999999, 2024-04-26T04:32:36.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20229604500000001, 2024-04-26T04:32:36.030 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.030 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.030 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:36.030 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.049630+0000", 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.049630+0000", 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.049633+0000", 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0199999999999999e-06 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.049646+0000", 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3596e-05 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.049678+0000", 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1754999999999997e-05 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.049694+0000", 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5860000000000001e-05 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.149935+0000", 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10024077000000001 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.150082+0000", 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000147658 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.251926+0000", 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10184338599999999 2024-04-26T04:32:36.033 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.034 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.034 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.034 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.7s3 36/30 ECSubWrite(tid=72, reqid=client.4670.0:270, at_version=36'8, trim_to=0'0, roll_forward_to=34'6))", 2024-04-26T04:32:36.034 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:30.229324+0000", 2024-04-26T04:32:36.034 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.7838669710000001, 2024-04-26T04:32:36.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.201857228, 2024-04-26T04:32:36.034 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.034 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.034 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:36.034 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.229324+0000", 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.229324+0000", 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.229326+0000", 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7099999999999999e-06 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.229375+0000", 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9713999999999999e-05 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.229434+0000", 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9154000000000003e-05 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.329570+0000", 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100135223 2024-04-26T04:32:36.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.329730+0000", 2024-04-26T04:32:36.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015983800000000001 2024-04-26T04:32:36.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:36.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.329755+0000", 2024-04-26T04:32:36.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5017000000000001e-05 2024-04-26T04:32:36.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:36.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.431071+0000", 2024-04-26T04:32:36.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101316064 2024-04-26T04:32:36.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.431181+0000", 2024-04-26T04:32:36.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011050800000000001 2024-04-26T04:32:36.038 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.038 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.038 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.038 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:320 3.6s0 3:647b920d:::smithi17829308-33:head [create,setxattr eROReghfPdlf^m`e`^hhNZXOkdsQ]]kX`senUiSZ (875) in=40b,setxattr s`rNbaqcP\\diTaoZYXg[m[inZ`]QtZdjom`_dhUX (1928) in=40b,setxattr [[YXf (69) in=5b,setxattr ]rjrtd]YkcY\\lQ[bOp_lNWs\\NOirOQ[Ri`rndifj (5260) in=40b,setxattr n[ZSX[Rc`\\dlYpV^rPdsQfnbg^l]ioSo^`oZsXaf (19016) in=40b,setxattr b\\ffPRnhZeR\\shWQYPRaZVroNXYoUYf[eemPeQen (1648) in=40b,setxattr [gNas\\_VhjX_YRX[aQbekgbWYg`P_SWgskeiZbkP (1333) in=40b,setxattr Ysi^k[[WWsYagaeecamn^bcecbfS]p^i^jfgsmTY (6736) in=40b,setxattr Nm_N]c]XYfmpWd\\aiW^bPSdY]TjNrnirngZdd_jb (103) in=40b,setxattr cisVjO^iia\\[tQaXjaroW`VRNl[aiYR\\tcnhVsfb (8841) in=40b,setxattr Zgg]\\aOQfmaVkY (69) in=14b,setxattr X`i_`tsrZ[XRfqacNap`[O`nSf]O`[Z[Oc[aenTc (1647) in=40b,setxattr oV[OofncW[WdO[XrprmdpkoptWZRfcPm^Rkernnf (34) in=40b,setxattr \\UQYgqaahhcRrkP`h`SgamahYafPmsojP_snXkmc (314) in=40b,setxattr pjilgXb]Qkl]gt__a_ZrtRptOPONOVrsmqcZ__^T (840) in=40b,setxattr Wa`ZaT[b]kbqdZrfSUkVafjqXod\\qd (69) in=30b,setxattr [YaXgtehetrqtspbYYY[ZqritVsbZeW[ak`ma``l (418) in=40b,setxattr hPanNtTRrfi`tYcdO]eRl]`fj`hllRNnXThl\\YO (69) in=39b,setxattr S_`Ri[Qj]fZqXkqqZYcte[N]Wq\\cQ]PlQdllcb]s (1542) in=40b,setxattr ttkZc`XVXmWiqdYWeagrmr^i[OUs^jpafRbOUWdp (594) in=40b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:36.038 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:34.392445+0000", 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.6207456279999999, 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.331045494, 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 320 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392445+0000", 2024-04-26T04:32:36.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392445+0000", 2024-04-26T04:32:36.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392450+0000", 2024-04-26T04:32:36.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6569999999999996e-06 2024-04-26T04:32:36.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392623+0000", 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017330500000000001 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392664+0000", 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0399999999999999e-05 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.492771+0000", 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10010693 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.520054+0000", 2024-04-26T04:32:36.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.027283827 2024-04-26T04:32:36.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:36.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.520231+0000", 2024-04-26T04:32:36.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017687399999999999 2024-04-26T04:32:36.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:36.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.521930+0000", 2024-04-26T04:32:36.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0016988629999999999 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.722167+0000", 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20023658899999999 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.722468+0000", 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030164599999999998 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.723491+0000", 2024-04-26T04:32:36.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010224030000000001 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 40/30 ECSubWriteReply(tid=14, last_complete=40'7, committed=1, applied=1))", 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:34.525241+0000", 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.4879503760000001, 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.19654977900000001, 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.525241+0000", 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.525241+0000", 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.525242+0000", 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.06e-06 2024-04-26T04:32:36.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.525247+0000", 2024-04-26T04:32:36.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9279999999999998e-06 2024-04-26T04:32:36.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.525264+0000", 2024-04-26T04:32:36.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6133e-05 2024-04-26T04:32:36.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.525272+0000", 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2979999999999992e-06 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.721752+0000", 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.196480023 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.721790+0000", 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8337000000000002e-05 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:339 3.6s0 3:674cc614:::smithi17829308-50:head [append 0~425984] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.167739+0000", 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.84545163800000001, 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.32164745300000003, 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 339 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.048 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.167739+0000", 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.167739+0000", 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.167742+0000", 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7199999999999998e-06 2024-04-26T04:32:36.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.168395+0000", 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00065334500000000005 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.168429+0000", 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3481e-05 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.168448+0000", 2024-04-26T04:32:36.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9369999999999999e-05 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.168583+0000", 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013446700000000001 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.168741+0000", 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015873499999999999 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.170574+0000", 2024-04-26T04:32:36.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018325430000000001 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.172903+0000", 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0023287770000000002 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.489262+0000", 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.31635918400000002 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.489387+0000", 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000124831 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 40/30 ECSubWriteReply(tid=16, last_complete=40'9, committed=1, applied=1))", 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.173546+0000", 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.839644685, 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.113781411, 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.173546+0000", 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.173546+0000", 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.173548+0000", 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5030000000000001e-06 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.173556+0000", 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7129999999999994e-06 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.173581+0000", 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4703999999999999e-05 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.273688+0000", 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100107189 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287293+0000", 2024-04-26T04:32:36.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.013604824999999999 2024-04-26T04:32:36.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287328+0000", 2024-04-26T04:32:36.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4477000000000002e-05 2024-04-26T04:32:36.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.056 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:340 3.6s0 3:674cc614:::smithi17829308-50:head [append 0~573440 in=573440b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.177236+0000", 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.83595447199999995, 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50180553100000003, 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 340 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.177236+0000", 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.177236+0000", 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.177239+0000", 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5100000000000001e-06 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178007+0000", 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00076857200000000003 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178041+0000", 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3909999999999999e-05 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178057+0000", 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5906999999999999e-05 2024-04-26T04:32:36.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178187+0000", 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012976099999999999 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178351+0000", 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016396800000000001 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.179822+0000", 2024-04-26T04:32:36.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001470507 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.185617+0000", 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0057954089999999996 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.678895+0000", 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49327816899999999 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679042+0000", 2024-04-26T04:32:36.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000146818 2024-04-26T04:32:36.062 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.062 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.062 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.062 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:341 3.6s0 3:674cc614:::smithi17829308-50:head [append 0~196608 in=196608b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:36.062 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180077+0000", 2024-04-26T04:32:36.062 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.833113408, 2024-04-26T04:32:36.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49926801199999998, 2024-04-26T04:32:36.062 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.062 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.062 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:36.062 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 341 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180077+0000", 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180077+0000", 2024-04-26T04:32:36.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180081+0000", 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2459999999999999e-06 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180497+0000", 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00041676000000000001 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180529+0000", 2024-04-26T04:32:36.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1497e-05 2024-04-26T04:32:36.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180556+0000", 2024-04-26T04:32:36.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6594e-05 2024-04-26T04:32:36.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.181275+0000", 2024-04-26T04:32:36.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00071977499999999997 2024-04-26T04:32:36.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.181487+0000", 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211855 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.182884+0000", 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00139722 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287222+0000", 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10433723 2024-04-26T04:32:36.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679247+0000", 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.392025711 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679345+0000", 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8123999999999998e-05 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.068 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:342 3.6s0 3:674cc614:::smithi17829308-50:head [setxattr _header (56) in=63b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:36.068 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180608+0000", 2024-04-26T04:32:36.068 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.83258277400000003, 2024-04-26T04:32:36.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49908019100000001, 2024-04-26T04:32:36.068 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.068 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.068 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:36.068 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:36.068 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:36.068 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:36.069 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 342 2024-04-26T04:32:36.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.069 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180608+0000", 2024-04-26T04:32:36.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180608+0000", 2024-04-26T04:32:36.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180610+0000", 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8029999999999999e-06 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180619+0000", 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3170000000000008e-06 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180642+0000", 2024-04-26T04:32:36.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2442999999999998e-05 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180658+0000", 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6779999999999999e-05 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.183978+0000", 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003319598 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:36.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.184127+0000", 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014915100000000001 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.185029+0000", 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00090211999999999996 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287425+0000", 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10239564600000001 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:32:36.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679540+0000", 2024-04-26T04:32:36.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39211458199999999 2024-04-26T04:32:36.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679688+0000", 2024-04-26T04:32:36.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000148751 2024-04-26T04:32:36.073 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.073 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.073 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.073 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:343 3.6s0 3:674cc614:::smithi17829308-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180684+0000", 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.83250688699999997, 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.71138035399999999, 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 343 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180684+0000", 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180684+0000", 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180685+0000", 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2869999999999999e-06 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180691+0000", 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0929999999999999e-06 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180712+0000", 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0514e-05 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180728+0000", 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5665999999999999e-05 2024-04-26T04:32:36.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.286928+0000", 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.106200577 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287076+0000", 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014773100000000001 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.680466+0000", 2024-04-26T04:32:36.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39339001800000001 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.680567+0000", 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100756 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.892064+0000", 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.211497712 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 40/30 ECSubWriteReply(tid=19, last_complete=40'12, committed=1, applied=1))", 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.276045+0000", 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.73714579199999997, 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.111834502, 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276045+0000", 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276045+0000", 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276046+0000", 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5999999999999991e-07 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276052+0000", 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0870000000000003e-06 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276075+0000", 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2580000000000001e-05 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276085+0000", 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0667e-05 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.387860+0000", 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.111774675 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.387880+0000", 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9533000000000001e-05 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 40/30 ECSubWriteReply(tid=18, last_complete=40'11, committed=1, applied=1))", 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.276134+0000", 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.73705706800000004, 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.111811129, 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.082 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276134+0000", 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276134+0000", 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.083 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276136+0000", 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6899999999999999e-06 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276146+0000", 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.079e-05 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276170+0000", 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4090000000000001e-05 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276184+0000", 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.33e-05 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.387926+0000", 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.111742536 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.387945+0000", 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8723e-05 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.085 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.086 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.086 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 40/30 ECSubWriteReply(tid=19, last_complete=40'12, committed=1, applied=1))", 2024-04-26T04:32:36.086 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.276687+0000", 2024-04-26T04:32:36.086 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.73650436399999997, 2024-04-26T04:32:36.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.21146537600000001, 2024-04-26T04:32:36.086 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.086 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.086 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:36.086 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276687+0000", 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276687+0000", 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276688+0000", 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3629999999999999e-06 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.087 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276696+0000", 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.4169999999999992e-06 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.276720+0000", 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3263000000000001e-05 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.376802+0000", 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100082322 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.088 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.488116+0000", 2024-04-26T04:32:36.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.111314318 2024-04-26T04:32:36.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.089 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.089 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.488152+0000", 2024-04-26T04:32:36.089 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5692999999999999e-05 2024-04-26T04:32:36.089 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.089 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.089 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.089 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.089 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.089 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 40/30 ECSubWriteReply(tid=17, last_complete=40'10, committed=1, applied=1))", 2024-04-26T04:32:36.090 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.472632+0000", 2024-04-26T04:32:36.090 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.54055905400000004, 2024-04-26T04:32:36.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.116948988, 2024-04-26T04:32:36.090 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.090 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.090 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:36.090 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.090 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.090 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.472632+0000", 2024-04-26T04:32:36.090 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.090 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.090 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.093 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.093 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.472632+0000", 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.472632+0000", 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0299999999999999e-07 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.472637+0000", 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1540000000000004e-06 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.094 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.472652+0000", 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5532999999999998e-05 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.472661+0000", 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7299999999999994e-06 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.589558+0000", 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.116897212 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.095 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.589581+0000", 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2756e-05 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 40/30 ECSubWriteReply(tid=18, last_complete=40'11, committed=1, applied=1))", 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.472675+0000", 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.54051621100000002, 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.116976055, 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.096 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.472675+0000", 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.472675+0000", 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.097 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.472675+0000", 2024-04-26T04:32:36.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2e-07 2024-04-26T04:32:36.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.472681+0000", 2024-04-26T04:32:36.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3069999999999999e-06 2024-04-26T04:32:36.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.098 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.098 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.098 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.472694+0000", 2024-04-26T04:32:36.098 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.242e-05 2024-04-26T04:32:36.098 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.472703+0000", 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4299999999999995e-06 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.589631+0000", 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.116927348 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.589651+0000", 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0129999999999999e-05 2024-04-26T04:32:36.099 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 40/30 ECSubWriteReply(tid=19, last_complete=40'12, committed=1, applied=1))", 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.473689+0000", 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.53950200699999995, 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.11601336900000001, 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.100 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.473689+0000", 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.473689+0000", 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.473690+0000", 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.607e-06 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.473700+0000", 2024-04-26T04:32:36.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3829999999999998e-06 2024-04-26T04:32:36.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.473728+0000", 2024-04-26T04:32:36.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8433999999999999e-05 2024-04-26T04:32:36.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.473743+0000", 2024-04-26T04:32:36.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4524e-05 2024-04-26T04:32:36.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.589689+0000", 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.11594618900000001 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.589702+0000", 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3232e-05 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpRead(3.7s3 41/30 ECSubRead(tid=119, to_read={3:e1f05b66:::smithi17829308-31:head=0,4096,0}, subchunks={3:e1f05b66:::smithi17829308-31:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.678899+0000", 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.33429165300000002, 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.200839504, 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.678899+0000", 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.678899+0000", 2024-04-26T04:32:36.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.678900+0000", 2024-04-26T04:32:36.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6100000000000001e-07 2024-04-26T04:32:36.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.678909+0000", 2024-04-26T04:32:36.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4790000000000002e-06 2024-04-26T04:32:36.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.106 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.678921+0000", 2024-04-26T04:32:36.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2176e-05 2024-04-26T04:32:36.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.106 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.779039+0000", 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100117974 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.879248+0000", 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100209061 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.879739+0000", 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00049035300000000001 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.107 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.108 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.108 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.108 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:36.109 INFO:tasks.rados.rados.0.smithi178.stdout:137: finishing write tid 2 to smithi17829308-17 2024-04-26T04:32:36.109 INFO:tasks.rados.rados.0.smithi178.stdout:137: finishing write tid 3 to smithi17829308-17 2024-04-26T04:32:36.110 INFO:tasks.rados.rados.0.smithi178.stdout:137: finishing write tid 4 to smithi17829308-17 2024-04-26T04:32:36.110 INFO:tasks.rados.rados.0.smithi178.stdout:136: finishing write tid 3 to smithi17829308-15 2024-04-26T04:32:36.110 INFO:tasks.rados.rados.0.smithi178.stdout:136: finishing write tid 4 to smithi17829308-15 2024-04-26T04:32:36.110 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:36.107+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:36.110 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T04:32:36.206 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T04:32:36.244 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:36.244 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:32:36.244 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.244 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 41/30 ECSubWriteReply(tid=122, last_complete=41'38, committed=1, applied=1))", 2024-04-26T04:32:36.244 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:36.053403+0000", 2024-04-26T04:32:36.244 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.18912928500000001, 2024-04-26T04:32:36.244 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.189172388, 2024-04-26T04:32:36.244 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.244 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.244 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:36.244 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053403+0000", 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053403+0000", 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053404+0000", 2024-04-26T04:32:36.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.0699999999999996e-07 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053412+0000", 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3819999999999998e-06 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053430+0000", 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8314000000000001e-05 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053441+0000", 2024-04-26T04:32:36.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1007e-05 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 41/30 ECSubWriteReply(tid=122, last_complete=41'38, committed=1, applied=1))", 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:36.053480+0000", 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.189053147, 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.189183671, 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053480+0000", 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053480+0000", 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053480+0000", 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0200000000000002e-07 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053483+0000", 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.039e-06 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053492+0000", 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4029999999999996e-06 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053498+0000", 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.2939999999999996e-06 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.249 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 41/30 ECSubWriteReply(tid=122, last_complete=41'38, committed=1, applied=1))", 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:36.053553+0000", 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.18897982699999999, 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.18919525500000001, 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053553+0000", 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053553+0000", 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053554+0000", 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6599999999999995e-07 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053558+0000", 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2019999999999997e-06 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053569+0000", 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1395e-05 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053575+0000", 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.665e-06 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpReadReply(3.1s0 41/30 ECSubReadReply(tid=123, attrs_read=0))", 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:36.054040+0000", 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.188492827, 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.18874239400000001, 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.054040+0000", 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.054040+0000", 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.054041+0000", 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.54e-07 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.054054+0000", 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3407000000000001e-05 2024-04-26T04:32:36.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.054069+0000", 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.483e-05 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.154161+0000", 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100092167 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.255 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpReadReply(3.1s0 41/30 ECSubReadReply(tid=123, attrs_read=0))", 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:36.054682+0000", 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.18785022600000001, 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.18813148199999999, 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.054682+0000", 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.054682+0000", 2024-04-26T04:32:36.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.054684+0000", 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.547e-06 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.054703+0000", 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9366999999999999e-05 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.054729+0000", 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6123e-05 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.054742+0000", 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2754e-05 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:358 3.1s0 3:8f617591:::smithi17829308-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.339814+0000", 2024-04-26T04:32:36.258 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.90271877499999997, 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.90304179100000004, 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 358 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339814+0000", 2024-04-26T04:32:36.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339814+0000", 2024-04-26T04:32:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339815+0000", 2024-04-26T04:32:36.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2300000000000001e-07 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339817+0000", 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8729999999999999e-06 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339827+0000", 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2669999999999995e-06 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339834+0000", 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0539999999999997e-06 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339863+0000", 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.904e-05 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339867+0000", 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9779999999999999e-06 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053012+0000", 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.71314480999999996 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053102+0000", 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0791999999999994e-05 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 41/30 ECSubWriteReply(tid=122, last_complete=41'38, committed=1, applied=1))", 2024-04-26T04:32:36.263 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:36.086420+0000", 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.15611234900000001, 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.15649216699999999, 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.086420+0000", 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.086420+0000", 2024-04-26T04:32:36.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.086422+0000", 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.646e-06 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.086431+0000", 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9239999999999996e-06 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.086458+0000", 2024-04-26T04:32:36.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.711e-05 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.086472+0000", 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3893000000000001e-05 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpReadReply(3.1s0 41/30 ECSubReadReply(tid=123, attrs_read=0))", 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:36.150279+0000", 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.092253428999999998, 2024-04-26T04:32:36.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.092665866999999999, 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.150279+0000", 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.150279+0000", 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.150279+0000", 2024-04-26T04:32:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.150279+0000", 2024-04-26T04:32:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.150309+0000", 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9397999999999999e-05 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.150319+0000", 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0720000000000001e-05 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 41/30 ECSubWriteReply(tid=122, last_complete=41'38, committed=1, applied=1))", 2024-04-26T04:32:36.269 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:36.154298+0000", 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.088234989, 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.088679785999999997, 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "no flag points reached", 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.154298+0000", 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.154298+0000", 2024-04-26T04:32:36.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.154298+0000", 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4700000000000001e-07 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.154302+0000", 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.1819999999999999e-06 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:36.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.154311+0000", 2024-04-26T04:32:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7849999999999997e-06 2024-04-26T04:32:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:32:36.272 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 9 2024-04-26T04:32:36.272 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:36.272 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:36.240+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:36.273 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T04:32:36.293 INFO:tasks.rados.rados.0.smithi178.stdout:136: finishing write tid 5 to smithi17829308-15 2024-04-26T04:32:36.293 INFO:tasks.rados.rados.0.smithi178.stdout:136: oid 15 updating version 0 to 14 2024-04-26T04:32:36.294 INFO:tasks.rados.rados.0.smithi178.stdout:136: oid 15 updating version 14 to 16 2024-04-26T04:32:36.294 INFO:tasks.rados.rados.0.smithi178.stdout:136: oid 15 updating version 16 to 17 2024-04-26T04:32:36.294 INFO:tasks.rados.rados.0.smithi178.stdout:136: oid 15 version 17 is already newer than 15 2024-04-26T04:32:36.294 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 15 v 17 (ObjNum 72 snap 4 seq_num 72) dirty exists 2024-04-26T04:32:36.294 INFO:tasks.rados.rados.0.smithi178.stdout:136: left oid 15 (ObjNum 72 snap 4 seq_num 72) 2024-04-26T04:32:36.310 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:36.307+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:36.332 INFO:tasks.rados.rados.0.smithi178.stdout:130: done (8 left) 2024-04-26T04:32:36.332 INFO:tasks.rados.rados.0.smithi178.stdout:131: done (7 left) 2024-04-26T04:32:36.332 INFO:tasks.rados.rados.0.smithi178.stdout:132: done (6 left) 2024-04-26T04:32:36.332 INFO:tasks.rados.rados.0.smithi178.stdout:133: done (5 left) 2024-04-26T04:32:36.332 INFO:tasks.rados.rados.0.smithi178.stdout:134: done (4 left) 2024-04-26T04:32:36.333 INFO:tasks.rados.rados.0.smithi178.stdout:136: done (3 left) 2024-04-26T04:32:36.333 INFO:tasks.rados.rados.0.smithi178.stdout:138: done (2 left) 2024-04-26T04:32:36.333 INFO:tasks.rados.rados.0.smithi178.stdout:139: done (1 left) 2024-04-26T04:32:36.333 INFO:tasks.rados.rados.0.smithi178.stdout:140: read oid 33 snap -1 2024-04-26T04:32:36.333 INFO:tasks.rados.rados.0.smithi178.stdout:140: expect (ObjNum 32 snap 0 seq_num 32) 2024-04-26T04:32:36.333 INFO:tasks.rados.rados.0.smithi178.stdout:141: snap_create 2024-04-26T04:32:36.385 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:36.386 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:36.407 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:36.407 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:36.412 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:36.411+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:36.445 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:36.445 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:36.452 INFO:tasks.rados.rados.0.smithi178.stdout:137: finishing write tid 5 to smithi17829308-17 2024-04-26T04:32:36.452 INFO:tasks.rados.rados.0.smithi178.stdout:137: oid 17 updating version 0 to 36 2024-04-26T04:32:36.452 INFO:tasks.rados.rados.0.smithi178.stdout:137: oid 17 version 36 is already newer than 35 2024-04-26T04:32:36.452 INFO:tasks.rados.rados.0.smithi178.stdout:137: oid 17 updating version 36 to 38 2024-04-26T04:32:36.453 INFO:tasks.rados.rados.0.smithi178.stdout:137: oid 17 version 38 is already newer than 37 2024-04-26T04:32:36.453 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 17 v 38 (ObjNum 73 snap 4 seq_num 73) dirty exists 2024-04-26T04:32:36.453 INFO:tasks.rados.rados.0.smithi178.stdout:137: left oid 17 (ObjNum 73 snap 4 seq_num 73) 2024-04-26T04:32:36.480 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:36.481 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:36.481 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:32:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-04-26T04:32:36.481 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-04-26T04:32:36.481 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:36.497 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T04:32:36.498 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:36.498 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:36.512 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:36.510+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:36.518 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:36.519 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:36.539 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:36.539 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:36.561 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:36.561 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:36.577 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:36.577 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:36.598 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:36.598 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:36.614 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:36.608+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:36.614 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:36.614 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:36.619 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:36.619 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:36.629 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T04:32:36.641 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:36.642 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:36.660 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:36.660 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:36.682 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:36.682 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:36.706 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:36.706 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:36.715 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:36.710+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:36.726 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:36.726 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:36.740 INFO:tasks.rados.rados.0.smithi178.stdout:140: expect (ObjNum 32 snap 0 seq_num 32) 2024-04-26T04:32:36.759 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:36.759 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:36.774 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T04:32:36.816 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:36.814+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:36.917 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:36.917+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:36.942 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:36.942 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:36.956 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T04:32:37.050 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:37.049+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:37.119 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:37.114+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:37.130 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:37.145 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T04:32:37.234 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:37.233+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:37.318 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:37.319 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:37.322 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:37.320+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:37.333 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T04:32:37.335 INFO:tasks.rados.rados.0.smithi178.stdout:137: done (2 left) 2024-04-26T04:32:37.335 INFO:tasks.rados.rados.0.smithi178.stdout:140: done (1 left) 2024-04-26T04:32:37.335 INFO:tasks.rados.rados.0.smithi178.stdout:141: done (0 left) 2024-04-26T04:32:37.335 INFO:tasks.rados.rados.0.smithi178.stdout:142: snap_remove snap 0 2024-04-26T04:32:37.339 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-26T04:32:37.423 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:37.423+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:37.515 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:37.515 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:37.526 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:37.523+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:37.530 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T04:32:37.627 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:37.628+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:37.647 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:37.647 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:37.661 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T04:32:37.728 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:37.722+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:37.793 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:37.793 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:37.809 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T04:32:37.852 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:37.850+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:37.930 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:37.929+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:37.976 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:37.976 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:37.990 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T04:32:38.030 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:38.029+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:38.111 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:38.112 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:38.125 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T04:32:38.131 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:38.129+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:38.232 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:38.231+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:38.244 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:38.244 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:38.258 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T04:32:38.316 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [2] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T04:32:38.316 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-26T04:32:38.333 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:38.331+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:38.337 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-04-26T04:32:38.345 INFO:tasks.rados.rados.0.smithi178.stdout:142: done (0 left) 2024-04-26T04:32:38.345 INFO:tasks.rados.rados.0.smithi178.stdout:143: delete oid 1 current snap is 6 2024-04-26T04:32:38.379 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:38.380 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:38.394 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T04:32:38.434 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:38.433+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:38.518 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:38.519 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:38.532 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T04:32:38.534 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:38.533+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:38.636 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:38.630+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:38.649 INFO:tasks.rados.rados.0.smithi178.stdout:143: done (0 left) 2024-04-26T04:32:38.650 INFO:tasks.rados.rados.0.smithi178.stdout:144: snap_create 2024-04-26T04:32:38.656 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:38.656 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:38.672 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T04:32:38.737 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:38.735+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:38.745 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T04:32:38.838 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:38.836+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:38.843 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:38.843 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:38.858 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T04:32:38.906 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:38.907 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:38.938 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:38.937+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:38.980 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:38.980 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:38.995 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T04:32:39.040 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:39.038+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:39.075 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:39.075 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:39.103 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:39.103 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:39.120 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:39.120 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:39.130 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:39.131 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:39.134 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T04:32:39.140 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:39.137+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:39.188 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:39.188 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:39.220 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:39.220 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:39.241 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:39.238+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:39.242 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:39.242 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:39.266 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:39.267 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:39.290 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:39.290 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:39.301 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:39.301 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:39.315 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T04:32:39.316 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:39.316 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:39.341 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:39.341 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:39.342 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:39.336+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:39.351 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.0 to 1 (10000) 2024-04-26T04:32:39.362 INFO:tasks.rados.rados.0.smithi178.stdout:144: done (0 left) 2024-04-26T04:32:39.362 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 31 current snap is 7 2024-04-26T04:32:39.362 INFO:tasks.rados.rados.0.smithi178.stdout:145: seq_num 75 ranges {1687552=491520,2179072=770048} 2024-04-26T04:32:39.366 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:39.367 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:39.375 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-26T04:32:39.388 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:39.389 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:39.398 INFO:tasks.rados.rados.0.smithi178.stdout:145: writing smithi17829308-31 from 1687552 to 2179072 tid 1 2024-04-26T04:32:39.411 INFO:tasks.rados.rados.0.smithi178.stdout:145: writing smithi17829308-31 from 2179072 to 2949120 tid 2 2024-04-26T04:32:39.411 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 16 current snap is 7 2024-04-26T04:32:39.411 INFO:tasks.rados.rados.0.smithi178.stdout:146: seq_num 76 ranges {262144=507904,770048=507904,1277952=262144} 2024-04-26T04:32:39.421 INFO:tasks.rados.rados.0.smithi178.stdout:146: writing smithi17829308-16 from 262144 to 770048 tid 1 2024-04-26T04:32:39.427 INFO:tasks.rados.rados.0.smithi178.stdout:146: writing smithi17829308-16 from 770048 to 1277952 tid 2 2024-04-26T04:32:39.430 INFO:tasks.rados.rados.0.smithi178.stdout:146: writing smithi17829308-16 from 1277952 to 1540096 tid 3 2024-04-26T04:32:39.430 INFO:tasks.rados.rados.0.smithi178.stdout:147: copy_from oid 10 from oid 39 current snap is 7 2024-04-26T04:32:39.430 INFO:tasks.rados.rados.0.smithi178.stdout:148: read oid 48 snap 3 2024-04-26T04:32:39.431 INFO:tasks.rados.rados.0.smithi178.stdout:148: expect (ObjNum 47 snap 0 seq_num 47) 2024-04-26T04:32:39.431 INFO:tasks.rados.rados.0.smithi178.stdout:149: copy_from oid 50 from oid 1 current snap is 7 2024-04-26T04:32:39.431 INFO:tasks.rados.rados.0.smithi178.stdout:150: snap_remove snap 6 2024-04-26T04:32:39.442 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:39.438+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:39.477 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:39.477 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:39.481 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:39.481 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:32:39.481 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:32:39.481 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:32:39.482 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.482 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-26T04:32:39.482 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:32:39.482 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 30.007538309000001, 2024-04-26T04:32:39.482 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.261775152, 2024-04-26T04:32:39.482 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.482 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.482 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.482 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.482 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:39.482 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:39.482 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 1 2024-04-26T04:32:39.482 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.483 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.483 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.483 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.483 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:32:39.483 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.483 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.483 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.483 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.483 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:32:39.483 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.483 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.483 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.483 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.484 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471107+0000", 2024-04-26T04:32:39.484 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4540000000000001e-06 2024-04-26T04:32:39.484 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.484 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.484 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.484 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471120+0000", 2024-04-26T04:32:39.484 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3217999999999999e-05 2024-04-26T04:32:39.484 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.484 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.484 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.484 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471147+0000", 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6461e-05 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471188+0000", 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1539000000000003e-05 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.472842+0000", 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016540700000000001 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for peered", 2024-04-26T04:32:39.485 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.472873+0000", 2024-04-26T04:32:39.486 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0300000000000001e-05 2024-04-26T04:32:39.486 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.486 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.486 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.486 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.729650+0000", 2024-04-26T04:32:39.486 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2567770170000001 2024-04-26T04:32:39.486 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.486 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.486 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.486 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.729957+0000", 2024-04-26T04:32:39.486 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00030700500000000001 2024-04-26T04:32:39.486 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.486 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.730324+0000", 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00036723099999999998 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732599+0000", 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0022751260000000001 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732706+0000", 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010741 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.487 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.488 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.488 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732804+0000", 2024-04-26T04:32:39.488 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7348000000000004e-05 2024-04-26T04:32:39.488 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.488 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.488 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.488 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732880+0000", 2024-04-26T04:32:39.488 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5972999999999999e-05 2024-04-26T04:32:39.488 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.488 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.488 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.488 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.488 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708719+0000", 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.769923327000001, 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60283476700000005, 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 115 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.489 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708719+0000", 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708719+0000", 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708720+0000", 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6700000000000003e-07 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.490 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.491 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.491 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708854+0000", 2024-04-26T04:32:39.491 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013397699999999999 2024-04-26T04:32:39.491 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.491 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.491 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.491 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708866+0000", 2024-04-26T04:32:39.491 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1875000000000001e-05 2024-04-26T04:32:39.491 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.491 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.491 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.491 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708874+0000", 2024-04-26T04:32:39.492 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0600000000000008e-06 2024-04-26T04:32:39.492 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.492 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.492 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.492 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.818243+0000", 2024-04-26T04:32:39.492 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10936897900000001 2024-04-26T04:32:39.492 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.492 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.492 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.492 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.818373+0000", 2024-04-26T04:32:39.492 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012997700000000001 2024-04-26T04:32:39.492 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.818585+0000", 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021234699999999999 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820901+0000", 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0023151669999999999 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311251+0000", 2024-04-26T04:32:39.493 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.49035071699999999 2024-04-26T04:32:39.494 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.494 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.494 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.494 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311421+0000", 2024-04-26T04:32:39.494 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016933399999999999 2024-04-26T04:32:39.494 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.494 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.494 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.494 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311554+0000", 2024-04-26T04:32:39.494 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013366699999999999 2024-04-26T04:32:39.494 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.494 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.494 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708888+0000", 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.769754583000001, 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60310785300000003, 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 116 2024-04-26T04:32:39.495 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.496 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.496 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.496 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.496 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708888+0000", 2024-04-26T04:32:39.496 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.496 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.496 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.496 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.496 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708888+0000", 2024-04-26T04:32:39.496 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.496 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.496 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.496 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.497 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708889+0000", 2024-04-26T04:32:39.497 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8700000000000005e-07 2024-04-26T04:32:39.497 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.497 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.497 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.497 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708894+0000", 2024-04-26T04:32:39.497 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.446e-06 2024-04-26T04:32:39.497 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.497 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.497 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.497 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708904+0000", 2024-04-26T04:32:39.497 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7059999999999999e-06 2024-04-26T04:32:39.497 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708912+0000", 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3730000000000004e-06 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.819534+0000", 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.110621994 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.819754+0000", 2024-04-26T04:32:39.498 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00022034999999999999 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820112+0000", 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00035776299999999998 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821816+0000", 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001704346 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311719+0000", 2024-04-26T04:32:39.499 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.48990303600000001 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311876+0000", 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015653800000000001 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311996+0000", 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012051400000000001 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.500 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.501 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.501 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:39.501 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:32:39.501 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.769720377999999, 2024-04-26T04:32:39.501 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60751630199999995, 2024-04-26T04:32:39.501 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.501 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.501 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.501 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.501 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:39.501 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:39.501 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 117 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.502 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.503 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708923+0000", 2024-04-26T04:32:39.503 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3300000000000002e-07 2024-04-26T04:32:39.503 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.503 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.503 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.503 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708929+0000", 2024-04-26T04:32:39.503 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.9959999999999999e-06 2024-04-26T04:32:39.503 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.503 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.503 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.503 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708941+0000", 2024-04-26T04:32:39.503 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2085e-05 2024-04-26T04:32:39.503 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708948+0000", 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.8170000000000001e-06 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820967+0000", 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.112019008 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821107+0000", 2024-04-26T04:32:39.504 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000139825 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821309+0000", 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020262899999999999 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.822530+0000", 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0012206129999999999 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:39.505 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316288+0000", 2024-04-26T04:32:39.506 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.49375772400000001 2024-04-26T04:32:39.506 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.506 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.506 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.506 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316375+0000", 2024-04-26T04:32:39.506 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7312999999999996e-05 2024-04-26T04:32:39.506 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.506 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.506 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.506 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316439+0000", 2024-04-26T04:32:39.506 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3658999999999995e-05 2024-04-26T04:32:39.506 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.506 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:161 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 20.739351023000001, 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51490446599999995, 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:39.507 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 161 2024-04-26T04:32:39.508 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.508 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.508 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.508 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.508 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:32:39.508 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.508 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.508 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.508 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.508 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:32:39.508 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.508 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.508 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.509 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.509 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:32:39.509 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6599999999999996e-07 2024-04-26T04:32:39.509 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.509 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.509 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.509 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739378+0000", 2024-04-26T04:32:39.509 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.5968000000000001e-05 2024-04-26T04:32:39.509 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.509 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.509 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.509 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739390+0000", 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1595e-05 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739398+0000", 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0029999999999994e-06 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.746760+0000", 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0073621420000000003 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.510 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.746876+0000", 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000115789 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.747088+0000", 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021234800000000001 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252667+0000", 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.50557901500000002 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.511 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254031+0000", 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001363518 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254124+0000", 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3376999999999997e-05 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254196+0000", 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2045000000000006e-05 2024-04-26T04:32:39.512 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739429+0000", 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 20.739213840000001, 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51514356699999997, 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:39.513 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:39.514 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 162 2024-04-26T04:32:39.514 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.514 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.514 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.514 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.514 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739429+0000", 2024-04-26T04:32:39.514 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.514 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.514 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.514 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.514 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739429+0000", 2024-04-26T04:32:39.514 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.514 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739430+0000", 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8999999999999995e-07 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739435+0000", 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.5969999999999999e-06 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739446+0000", 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0535e-05 2024-04-26T04:32:39.515 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739453+0000", 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.4800000000000004e-06 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.943028+0000", 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20357512 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.943276+0000", 2024-04-26T04:32:39.516 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024770199999999998 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.943640+0000", 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00036409399999999997 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252675+0000", 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30903449199999999 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:39.517 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254318+0000", 2024-04-26T04:32:39.518 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016429179999999999 2024-04-26T04:32:39.518 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.518 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.518 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.518 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254463+0000", 2024-04-26T04:32:39.518 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000145623 2024-04-26T04:32:39.518 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.518 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.518 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.518 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254573+0000", 2024-04-26T04:32:39.518 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010921600000000001 2024-04-26T04:32:39.518 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.518 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.519 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.519 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.519 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.519 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:39.519 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:32:39.519 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 20.739177178999999, 2024-04-26T04:32:39.519 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.515346639, 2024-04-26T04:32:39.519 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.519 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.519 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.519 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.519 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 163 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.520 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.1699999999999997e-07 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739471+0000", 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.9219999999999996e-06 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739487+0000", 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5435999999999999e-05 2024-04-26T04:32:39.521 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739494+0000", 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2899999999999997e-06 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.944605+0000", 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20511091200000001 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.944825+0000", 2024-04-26T04:32:39.522 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00022035499999999999 2024-04-26T04:32:39.523 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.523 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.523 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:39.523 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.945203+0000", 2024-04-26T04:32:39.523 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00037760499999999999 2024-04-26T04:32:39.523 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.523 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.523 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:39.523 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252681+0000", 2024-04-26T04:32:39.523 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.307478311 2024-04-26T04:32:39.523 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.523 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.523 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:39.524 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254680+0000", 2024-04-26T04:32:39.524 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0019988129999999999 2024-04-26T04:32:39.524 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.524 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.524 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.524 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254756+0000", 2024-04-26T04:32:39.524 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5914999999999994e-05 2024-04-26T04:32:39.524 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.524 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.524 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.524 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254812+0000", 2024-04-26T04:32:39.524 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.6363000000000002e-05 2024-04-26T04:32:39.524 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 20.739137041999999, 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.61583993999999997, 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:39.525 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:39.526 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 164 2024-04-26T04:32:39.526 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.526 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.526 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.526 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.526 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:32:39.526 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.526 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.526 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.526 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.526 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:32:39.526 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.526 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0399999999999995e-07 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739594+0000", 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7046000000000007e-05 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739605+0000", 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1576000000000001e-05 2024-04-26T04:32:39.527 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739613+0000", 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1459999999999996e-06 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.046981+0000", 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30736738000000002 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.047229+0000", 2024-04-26T04:32:39.528 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024846999999999998 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.047619+0000", 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00038963799999999999 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254219+0000", 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20659982700000001 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:39.529 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355048+0000", 2024-04-26T04:32:39.530 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100829484 2024-04-26T04:32:39.530 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.530 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.530 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.530 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355225+0000", 2024-04-26T04:32:39.530 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017717399999999999 2024-04-26T04:32:39.530 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.530 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.530 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.530 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355346+0000", 2024-04-26T04:32:39.530 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000120495 2024-04-26T04:32:39.530 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.530 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 20.739015526999999, 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.71903434899999996, 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:32:39.531 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 165 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.532 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739628+0000", 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7000000000000004e-07 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739645+0000", 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6901999999999998e-05 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739656+0000", 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0556e-05 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.533 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739667+0000", 2024-04-26T04:32:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1535000000000001e-05 2024-04-26T04:32:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355474+0000", 2024-04-26T04:32:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.61580721299999996 2024-04-26T04:32:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355612+0000", 2024-04-26T04:32:39.534 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000137687 2024-04-26T04:32:39.535 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.535 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.535 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:32:39.535 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355816+0000", 2024-04-26T04:32:39.535 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020451900000000001 2024-04-26T04:32:39.535 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.535 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.535 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:32:39.535 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.357276+0000", 2024-04-26T04:32:39.535 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0014599179999999999 2024-04-26T04:32:39.535 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.535 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.535 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:32:39.536 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458392+0000", 2024-04-26T04:32:39.536 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101115921 2024-04-26T04:32:39.536 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.536 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.536 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.536 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458547+0000", 2024-04-26T04:32:39.536 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015461199999999999 2024-04-26T04:32:39.536 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.536 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.536 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.536 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458662+0000", 2024-04-26T04:32:39.536 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000114716 2024-04-26T04:32:39.536 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:31 3.es0 3:7950d065:::smithi17829308-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.211346+0000", 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 14.267296379999999, 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80728777299999999, 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:39.537 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:39.538 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 31 2024-04-26T04:32:39.538 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.538 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.538 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.538 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.538 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211346+0000", 2024-04-26T04:32:39.538 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.538 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.538 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.538 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.538 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211346+0000", 2024-04-26T04:32:39.538 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.538 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.539 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.539 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.539 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211350+0000", 2024-04-26T04:32:39.539 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.3400000000000002e-06 2024-04-26T04:32:39.539 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.540 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.540 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.540 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211978+0000", 2024-04-26T04:32:39.540 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00062828600000000004 2024-04-26T04:32:39.540 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.540 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.540 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.540 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212008+0000", 2024-04-26T04:32:39.540 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0287999999999999e-05 2024-04-26T04:32:39.540 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212024+0000", 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5316e-05 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312296+0000", 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100272797 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312539+0000", 2024-04-26T04:32:39.541 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000242542 2024-04-26T04:32:39.542 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.542 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.542 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:39.542 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.313858+0000", 2024-04-26T04:32:39.542 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001319312 2024-04-26T04:32:39.542 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.542 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.542 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:39.542 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319757+0000", 2024-04-26T04:32:39.542 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.005898927 2024-04-26T04:32:39.542 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.542 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.542 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018515+0000", 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69875785199999996 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018634+0000", 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000119113 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:32 3.es0 3:7950d065:::smithi17829308-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:39.543 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.212058+0000", 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 14.266584372000001, 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80687460799999999, 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 32 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.544 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212058+0000", 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212058+0000", 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212060+0000", 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2130000000000001e-06 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.545 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212065+0000", 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.5260000000000001e-06 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212084+0000", 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8399000000000001e-05 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212114+0000", 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0598000000000001e-05 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.546 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.314776+0000", 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10266165200000001 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.315053+0000", 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00027713999999999999 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316307+0000", 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00125442 2024-04-26T04:32:39.547 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321719+0000", 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.005411463 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018816+0000", 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69709755699999998 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018933+0000", 2024-04-26T04:32:39.548 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00011663999999999999 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:43 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~245760,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.217138+0000", 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 14.261504333, 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80275945999999998, 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:39.549 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:39.550 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 43 2024-04-26T04:32:39.550 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.550 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.550 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.550 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.550 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217138+0000", 2024-04-26T04:32:39.550 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.550 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.550 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.550 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.550 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217138+0000", 2024-04-26T04:32:39.550 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.550 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217141+0000", 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3499999999999999e-06 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217563+0000", 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00042241199999999999 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217591+0000", 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7501999999999999e-05 2024-04-26T04:32:39.551 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217605+0000", 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3866000000000001e-05 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.317118+0000", 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.099513764000000005 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.317307+0000", 2024-04-26T04:32:39.552 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018885199999999999 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.318924+0000", 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001616691 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321728+0000", 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0028038249999999998 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.553 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.019782+0000", 2024-04-26T04:32:39.554 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69805386400000002 2024-04-26T04:32:39.554 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.554 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.554 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.554 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.019898+0000", 2024-04-26T04:32:39.554 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000116334 2024-04-26T04:32:39.554 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.554 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.554 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.554 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.554 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.554 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:44 3.es0 3:7dd262f9:::smithi17829308-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:39.554 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.217633+0000", 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 14.261009872000001, 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80280496999999995, 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 44 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.555 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217633+0000", 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217633+0000", 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217634+0000", 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.06e-06 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.556 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217640+0000", 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.614e-06 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217654+0000", 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4279e-05 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217662+0000", 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2959999999999999e-06 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.557 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319858+0000", 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102196092 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319985+0000", 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012646000000000001 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.320976+0000", 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00099148399999999994 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.558 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.322066+0000", 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001090246 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.020239+0000", 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69817257799999999 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.020438+0000", 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00019886099999999999 2024-04-26T04:32:39.559 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:267 3.2s0 3:4497087f:::smithi17829308-22:head [read 0~360448,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.131109+0000", 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 9.3475337540000005, 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70452207899999997, 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:39.560 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 267 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131109+0000", 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131109+0000", 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.561 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131111+0000", 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9700000000000002e-06 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131119+0000", 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4600000000000003e-06 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131136+0000", 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6820000000000002e-05 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.562 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.563 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.563 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131146+0000", 2024-04-26T04:32:39.563 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.2499999999999995e-06 2024-04-26T04:32:39.563 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.563 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.563 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.563 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131232+0000", 2024-04-26T04:32:39.563 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6907e-05 2024-04-26T04:32:39.563 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.563 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131352+0000", 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000119164 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.835631+0000", 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70427950800000005 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.564 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:268 3.2s0 3:4497087f:::smithi17829308-22:head [read 0~360448,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 9.3474742079999995, 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70524462600000004, 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 268 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.565 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3799999999999997e-07 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.566 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131173+0000", 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8929999999999998e-06 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131188+0000", 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5208000000000001e-05 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131197+0000", 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1950000000000003e-06 2024-04-26T04:32:39.567 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131969+0000", 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00077229900000000005 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.132116+0000", 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014686300000000001 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.836413+0000", 2024-04-26T04:32:39.568 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70429752999999995 2024-04-26T04:32:39.569 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.569 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.569 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.569 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.569 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.569 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:269 3.2s0 3:4497087f:::smithi17829308-22:head [read 0~360448,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:39.569 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.131236+0000", 2024-04-26T04:32:39.569 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 9.3474063170000008, 2024-04-26T04:32:39.569 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70573893899999995, 2024-04-26T04:32:39.569 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.569 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.570 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:39.570 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.570 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:39.570 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:39.570 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 269 2024-04-26T04:32:39.570 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.570 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.570 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.570 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.570 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131236+0000", 2024-04-26T04:32:39.570 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.570 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.570 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131236+0000", 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131237+0000", 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.9800000000000003e-07 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131241+0000", 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.2239999999999997e-06 2024-04-26T04:32:39.571 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131250+0000", 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.0629999999999995e-06 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231335+0000", 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10008484400000001 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231436+0000", 2024-04-26T04:32:39.572 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000100435 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231576+0000", 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014048000000000001 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.836975+0000", 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.605399295 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.573 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.574 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:273 3.2s0 3:466bd692:::smithi17829308-25:head [delete] snapc 2=[2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:39.574 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.231373+0000", 2024-04-26T04:32:39.574 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 9.2472693919999998, 2024-04-26T04:32:39.574 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60595961600000003, 2024-04-26T04:32:39.574 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.574 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.574 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.574 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.574 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:39.574 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:39.574 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 273 2024-04-26T04:32:39.574 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.574 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.575 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.575 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.575 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231373+0000", 2024-04-26T04:32:39.578 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.578 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.578 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.578 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.578 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231373+0000", 2024-04-26T04:32:39.578 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.578 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.578 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.578 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231375+0000", 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.539e-06 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231383+0000", 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5660000000000001e-06 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231394+0000", 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1739000000000001e-05 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.579 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.580 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231405+0000", 2024-04-26T04:32:39.580 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0266e-05 2024-04-26T04:32:39.580 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.580 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.580 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.580 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231995+0000", 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00059051299999999998 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.232097+0000", 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010197899999999999 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.233122+0000", 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00102451 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.581 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.234316+0000", 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0011942019999999999 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.837219+0000", 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60290286500000001 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.837333+0000", 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000114437 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.582 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:349 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~442368] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.325227+0000", 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.1534155200000002, 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.56835219299999995, 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:39.583 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 349 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.325227+0000", 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.325227+0000", 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.584 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.325231+0000", 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8500000000000004e-06 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.325808+0000", 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00057670400000000004 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.325852+0000", 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.3961000000000001e-05 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.585 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.586 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.586 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.426013+0000", 2024-04-26T04:32:39.586 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100160764 2024-04-26T04:32:39.586 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.586 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.586 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.586 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.426151+0000", 2024-04-26T04:32:39.586 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013824700000000001 2024-04-26T04:32:39.586 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.586 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.586 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op must wait for map", 2024-04-26T04:32:39.586 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.426163+0000", 2024-04-26T04:32:39.586 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2656e-05 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.580183+0000", 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.154019447 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.580443+0000", 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00025980799999999999 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:39.587 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.582459+0000", 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0020159050000000001 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.590550+0000", 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0080916990000000008 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.893438+0000", 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.302887772 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.588 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.893579+0000", 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014138000000000001 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:351 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~49152] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.427267+0000", 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.0513756169999997, 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.56700841899999999, 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:39.589 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.590 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:39.590 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:39.590 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 351 2024-04-26T04:32:39.590 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.590 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.590 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.590 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.590 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.427267+0000", 2024-04-26T04:32:39.590 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.590 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.590 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.590 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.590 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.427267+0000", 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.427269+0000", 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5960000000000001e-06 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.427330+0000", 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.1243999999999998e-05 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.591 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.427353+0000", 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3393000000000001e-05 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.527490+0000", 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100136091 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.587991+0000", 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.060501006000000003 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.592 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.588318+0000", 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000327125 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.589552+0000", 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001233998 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.591162+0000", 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016098900000000001 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.593 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.594 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:39.594 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.994158+0000", 2024-04-26T04:32:39.594 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.402995995 2024-04-26T04:32:39.594 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.594 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.594 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.594 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.994276+0000", 2024-04-26T04:32:39.594 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000118081 2024-04-26T04:32:39.594 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.594 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.594 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.594 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.594 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:353 3.es0 3:7dd262f9:::smithi17829308-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.627803+0000", 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 3.8508402369999999, 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.66700178399999999, 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 353 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:39.595 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627803+0000", 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627803+0000", 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627804+0000", 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.68e-06 2024-04-26T04:32:39.596 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627813+0000", 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4759999999999998e-06 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627833+0000", 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9809e-05 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627844+0000", 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1250000000000001e-05 2024-04-26T04:32:39.597 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.598 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.598 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.598 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.691576+0000", 2024-04-26T04:32:39.598 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.063732043000000002 2024-04-26T04:32:39.598 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.598 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.598 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:39.598 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.691730+0000", 2024-04-26T04:32:39.598 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015389900000000001 2024-04-26T04:32:39.598 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.598 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.598 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:39.598 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.995375+0000", 2024-04-26T04:32:39.599 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30364495600000002 2024-04-26T04:32:39.599 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.599 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.599 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:39.599 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.995640+0000", 2024-04-26T04:32:39.599 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026531100000000001 2024-04-26T04:32:39.599 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:39.599 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:39.599 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:39.599 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:36.294804+0000", 2024-04-26T04:32:39.599 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.29916436000000002 2024-04-26T04:32:39.599 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.599 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.600 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.600 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:39.600 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:39.600 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:39.601 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:39.601 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:39.602 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:39.602 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T04:32:39.603 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:39.604 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:39.596+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:39.703 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:39.698+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:39.736 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:39.736 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:39.736 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:32:39.736 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:39.750 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T04:32:39.804 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:39.802+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:39.873 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:39.873 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:39.873 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:32:39.873 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:32:39.873 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:39.887 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T04:32:39.905 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:39.902+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:40.006 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:40.004+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:40.013 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:40.013 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:42 1.0 e26/23 1:30306672:devicehealth::main.db.0000000000000000:head v 26'28, mlcod=26'26)", 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.793093792000001, 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201587136, 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.014 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220676+0000", 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.843e-06 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.015 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220687+0000", 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1727e-05 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220715+0000", 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8033000000000001e-05 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.320857+0000", 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014171400000001 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.016 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.421143+0000", 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100285902 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.421175+0000", 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2413999999999998e-05 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.422168+0000", 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0009931269999999999 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.017 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.018 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.422261+0000", 2024-04-26T04:32:40.018 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2375999999999998e-05 2024-04-26T04:32:40.018 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.018 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.018 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.018 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.019 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.020 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:55 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'33, mlcod=27'29)", 2024-04-26T04:32:40.020 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:32:40.020 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.469703929000001, 2024-04-26T04:32:40.020 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.169473288, 2024-04-26T04:32:40.020 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.020 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.020 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.020 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.020 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.020 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.020 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544065+0000", 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.77e-06 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.021 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544195+0000", 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012971599999999999 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544219+0000", 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3476e-05 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.644316+0000", 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100097189 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.022 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.711737+0000", 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.067420846000000006 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.711770+0000", 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2959000000000001e-05 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.713428+0000", 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016584130000000001 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.023 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.713537+0000", 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010891900000000001 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:108 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'70, mlcod=27'64)", 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 23.205309765999999, 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30260933299999998, 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.024 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.025 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.025 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.025 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.025 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:32:40.025 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.025 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.025 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.025 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.025 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:32:40.025 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.025 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.025 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.37e-07 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808464+0000", 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8599999999999998e-06 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808474+0000", 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.9830000000000008e-06 2024-04-26T04:32:40.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908648+0000", 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10017390299999999 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908808+0000", 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000159576 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908845+0000", 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7026000000000001e-05 2024-04-26T04:32:40.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110983+0000", 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20213835199999999 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111067+0000", 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3996e-05 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.028 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:109 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'71, mlcod=27'64)", 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 23.105075042999999, 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202472819, 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.029 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908694+0000", 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.144e-06 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908751+0000", 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7268999999999997e-05 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.030 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908769+0000", 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8363e-05 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908780+0000", 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.096e-05 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.909341+0000", 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00056057500000000001 2024-04-26T04:32:40.031 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.909371+0000", 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0357000000000001e-05 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111074+0000", 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20170292500000001 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111165+0000", 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.1225999999999997e-05 2024-04-26T04:32:40.032 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.033 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.033 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.033 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.033 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:110 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'72, mlcod=27'64)", 2024-04-26T04:32:40.033 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:32:40.033 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 23.104940151000001, 2024-04-26T04:32:40.033 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20385624999999999, 2024-04-26T04:32:40.033 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.033 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.033 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.033 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.034 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.034 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.034 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:32:40.034 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.034 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.034 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.034 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:32:40.034 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.034 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.034 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908828+0000", 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9400000000000005e-07 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908836+0000", 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1929999999999993e-06 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908847+0000", 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0703000000000001e-05 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.035 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908855+0000", 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7829999999999999e-06 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009054+0000", 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10019966900000001 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009089+0000", 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4433e-05 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.036 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112582+0000", 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103492973 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112684+0000", 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000101802 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:111 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'73, mlcod=27'64)", 2024-04-26T04:32:40.037 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 23.104901220999999, 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.205422306, 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.038 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908867+0000", 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.03e-07 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908871+0000", 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5800000000000002e-06 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.039 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908883+0000", 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1507e-05 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009022+0000", 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100139593 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.109703+0000", 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100680741 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.040 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.109738+0000", 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5296000000000002e-05 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114207+0000", 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0044682239999999998 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114289+0000", 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2062000000000001e-05 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:112 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'74, mlcod=27'64)", 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 23.004707099000001, 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105317487, 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.042 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.043 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.043 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.043 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:32:40.043 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.043 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.043 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.043 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.043 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009061+0000", 2024-04-26T04:32:40.043 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.057e-06 2024-04-26T04:32:40.043 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.043 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.043 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.043 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009128+0000", 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6859999999999993e-05 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009147+0000", 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8196e-05 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009157+0000", 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0743e-05 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.044 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110254+0000", 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101096941 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110311+0000", 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6718999999999997e-05 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114296+0000", 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0039847709999999998 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.046 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.046 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114378+0000", 2024-04-26T04:32:40.046 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2200000000000006e-05 2024-04-26T04:32:40.046 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.046 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.046 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.046 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.046 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:114 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'76, mlcod=27'64)", 2024-04-26T04:32:40.046 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:32:40.046 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 23.004557850000001, 2024-04-26T04:32:40.046 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105329463, 2024-04-26T04:32:40.046 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.047 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.047 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.047 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.047 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.047 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.047 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:32:40.047 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.047 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.047 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.048 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.048 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:32:40.048 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.048 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.048 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.048 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:32:40.048 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.53e-07 2024-04-26T04:32:40.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.048 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.048 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.048 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009215+0000", 2024-04-26T04:32:40.048 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7099999999999998e-06 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009223+0000", 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5599999999999994e-06 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009229+0000", 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0800000000000002e-06 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.049 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111865+0000", 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102636071 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111941+0000", 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5615e-05 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114468+0000", 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002526875 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.050 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114539+0000", 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1198999999999994e-05 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:115 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'77, mlcod=27'64)", 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 23.004529977000001, 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105390922, 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.051 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.34e-07 2024-04-26T04:32:40.052 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009293+0000", 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4954999999999998e-05 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009304+0000", 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1263999999999999e-05 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009311+0000", 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6630000000000004e-06 2024-04-26T04:32:40.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112802+0000", 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10349130600000001 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112843+0000", 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0472999999999997e-05 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114547+0000", 2024-04-26T04:32:40.054 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001704552 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114628+0000", 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1374999999999999e-05 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:116 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'78, mlcod=27'64)", 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 23.004446724000001, 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105877241, 2024-04-26T04:32:40.055 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.056 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.46e-07 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009326+0000", 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5040000000000004e-06 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009334+0000", 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1929999999999993e-06 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009340+0000", 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.3729999999999998e-06 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.113595+0000", 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104254889 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.113649+0000", 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3887000000000001e-05 2024-04-26T04:32:40.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115105+0000", 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014560969999999999 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115198+0000", 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2851999999999998e-05 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.059 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.060 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:117 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'79, mlcod=27'64)", 2024-04-26T04:32:40.060 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:32:40.060 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 23.004419241000001, 2024-04-26T04:32:40.060 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10639752, 2024-04-26T04:32:40.060 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.060 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.060 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.060 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.060 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.060 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.060 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:32:40.060 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009349+0000", 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7300000000000002e-07 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.061 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009353+0000", 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5970000000000002e-06 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009361+0000", 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0600000000000008e-06 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009381+0000", 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9533000000000001e-05 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.062 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114709+0000", 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105328753 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114745+0000", 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5976000000000002e-05 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115683+0000", 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00093774899999999998 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.063 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115746+0000", 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2578999999999998e-05 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:155 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'109, mlcod=27'105)", 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.174231467999999, 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20562082700000001, 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.064 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839537+0000", 2024-04-26T04:32:40.065 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7400000000000003e-07 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839638+0000", 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000101365 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839653+0000", 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4874e-05 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.066 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939797+0000", 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014414100000001 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.040092+0000", 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100295305 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.040151+0000", 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8295999999999999e-05 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.067 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045057+0000", 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0049061390000000003 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045157+0000", 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000100133 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:156 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'110, mlcod=27'105)", 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.073935990999999, 2024-04-26T04:32:40.068 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10750937100000001, 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.069 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939833+0000", 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1030000000000001e-06 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939849+0000", 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6005999999999999e-05 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939866+0000", 2024-04-26T04:32:40.070 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7513e-05 2024-04-26T04:32:40.071 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.071 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.071 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.071 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939876+0000", 2024-04-26T04:32:40.072 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0154e-05 2024-04-26T04:32:40.072 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.072 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.072 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.072 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.041330+0000", 2024-04-26T04:32:40.072 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101453864 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.041398+0000", 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7617e-05 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047245+0000", 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058470140000000002 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.073 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047341+0000", 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6100000000000005e-05 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:157 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'111, mlcod=27'105)", 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.073835851999998, 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107487887, 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.074 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2300000000000001e-07 2024-04-26T04:32:40.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939942+0000", 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4369999999999996e-06 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939952+0000", 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.075e-05 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939963+0000", 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0069e-05 2024-04-26T04:32:40.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.042203+0000", 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102240215 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.042259+0000", 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5955999999999998e-05 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047353+0000", 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0050939080000000003 2024-04-26T04:32:40.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047420+0000", 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6928999999999996e-05 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:158 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'112, mlcod=27'105)", 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.073793265999999, 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.209867842, 2024-04-26T04:32:40.078 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939975+0000", 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8599999999999999e-07 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940084+0000", 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000109746 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940099+0000", 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4392999999999999e-05 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.080 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940110+0000", 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1236999999999999e-05 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043002+0000", 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102892262 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043054+0000", 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1922999999999997e-05 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.081 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149774+0000", 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10672018599999999 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149842+0000", 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7708999999999998e-05 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:159 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'113, mlcod=27'105)", 2024-04-26T04:32:40.082 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.073644644000002, 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20976125000000001, 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:32:40.083 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7000000000000005e-07 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940131+0000", 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6369999999999999e-06 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940143+0000", 2024-04-26T04:32:40.084 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1603000000000001e-05 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940165+0000", 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2042999999999999e-05 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043777+0000", 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103612145 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.085 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043807+0000", 2024-04-26T04:32:40.086 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0562999999999999e-05 2024-04-26T04:32:40.086 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.086 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.086 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.086 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149845+0000", 2024-04-26T04:32:40.086 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10603802900000001 2024-04-26T04:32:40.086 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.086 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.086 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.086 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149884+0000", 2024-04-26T04:32:40.087 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8659999999999999e-05 2024-04-26T04:32:40.087 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.087 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.087 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.087 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.087 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.087 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:160 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'114, mlcod=27'105)", 2024-04-26T04:32:40.087 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:32:40.087 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.073594138000001, 2024-04-26T04:32:40.087 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20974973299999999, 2024-04-26T04:32:40.087 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.087 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.087 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.088 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.088 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.088 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.088 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:32:40.088 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.088 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.088 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.088 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.088 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:32:40.088 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.088 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.088 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940174+0000", 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8999999999999998e-07 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940183+0000", 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6970000000000005e-06 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940194+0000", 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0983e-05 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.089 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940201+0000", 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4400000000000002e-06 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.044326+0000", 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104124984 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.044377+0000", 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1249999999999999e-05 2024-04-26T04:32:40.090 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149886+0000", 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10550875999999999 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149923+0000", 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7329000000000001e-05 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.091 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:161 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'115, mlcod=27'105)", 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.073558195, 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.209755314, 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.092 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940210+0000", 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.67e-07 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940330+0000", 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012061900000000001 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.093 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940344+0000", 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.367e-05 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940355+0000", 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.083e-05 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045239+0000", 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104883873 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.094 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045278+0000", 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9904000000000003e-05 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149925+0000", 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104646221 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149965+0000", 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9929999999999999e-05 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.095 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:162 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'116, mlcod=27'105)", 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.070094463, 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.206336451, 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:32:40.096 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943675+0000", 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8640000000000001e-06 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.097 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943692+0000", 2024-04-26T04:32:40.098 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7503000000000002e-05 2024-04-26T04:32:40.098 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.098 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.098 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.098 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943720+0000", 2024-04-26T04:32:40.098 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7569999999999999e-05 2024-04-26T04:32:40.098 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.098 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.098 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.098 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943734+0000", 2024-04-26T04:32:40.098 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3596e-05 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.046517+0000", 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102783897 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.046569+0000", 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1706000000000003e-05 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149968+0000", 2024-04-26T04:32:40.099 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10339902199999999 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150009+0000", 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1292999999999999e-05 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:163 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'117, mlcod=27'105)", 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.068593393, 2024-04-26T04:32:40.100 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20488266699999999, 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.101 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945176+0000", 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8470000000000001e-06 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945193+0000", 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7013000000000001e-05 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945221+0000", 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7670000000000001e-05 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.102 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945238+0000", 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7113e-05 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.147614+0000", 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202375893 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.147669+0000", 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5729000000000001e-05 2024-04-26T04:32:40.103 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150011+0000", 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002342083 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150057+0000", 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5318999999999998e-05 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.104 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:40.105 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T04:32:40.106 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:40.105+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:40.207 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:40.206+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:40.243 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:40.243 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:40.243 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:32:40.243 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:40.257 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T04:32:40.308 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:40.306+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:40.364 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.1 to 1 (10000) 2024-04-26T04:32:40.370 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-26T04:32:40.371 INFO:tasks.rados.rados.0.smithi178.stdout:150: done (5 left) 2024-04-26T04:32:40.371 INFO:tasks.rados.rados.0.smithi178.stdout:151: delete oid 18 current snap is 7 2024-04-26T04:32:40.371 INFO:tasks.rados.rados.0.smithi178.stdout:149: finishing copy_from to smithi17829308-50 2024-04-26T04:32:40.371 INFO:tasks.rados.rados.0.smithi178.stdout:149: got expected ENOENT (src dne) 2024-04-26T04:32:40.371 INFO:tasks.rados.rados.0.smithi178.stdout:149: finishing copy_from racing read to smithi17829308-50 2024-04-26T04:32:40.371 INFO:tasks.rados.rados.0.smithi178.stdout:145: finishing write tid 1 to smithi17829308-31 2024-04-26T04:32:40.371 INFO:tasks.rados.rados.0.smithi178.stdout:145: finishing write tid 2 to smithi17829308-31 2024-04-26T04:32:40.372 INFO:tasks.rados.rados.0.smithi178.stdout:145: finishing write tid 3 to smithi17829308-31 2024-04-26T04:32:40.372 INFO:tasks.rados.rados.0.smithi178.stdout:146: finishing write tid 1 to smithi17829308-16 2024-04-26T04:32:40.372 INFO:tasks.rados.rados.0.smithi178.stdout:148: expect (ObjNum 47 snap 0 seq_num 47) 2024-04-26T04:32:40.382 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:40.382 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:40.382 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:32:40.382 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:32:40.382 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:40.389 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-26T04:32:40.396 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T04:32:40.409 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:40.406+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:40.427 INFO:tasks.rados.rados.0.smithi178.stdout:146: finishing write tid 2 to smithi17829308-16 2024-04-26T04:32:40.428 INFO:tasks.rados.rados.0.smithi178.stdout:146: finishing write tid 3 to smithi17829308-16 2024-04-26T04:32:40.428 INFO:tasks.rados.rados.0.smithi178.stdout:146: finishing write tid 4 to smithi17829308-16 2024-04-26T04:32:40.428 INFO:tasks.rados.rados.0.smithi178.stdout:147: finishing copy_from racing read to smithi17829308-10 2024-04-26T04:32:40.428 INFO:tasks.rados.rados.0.smithi178.stdout:145: finishing write tid 4 to smithi17829308-31 2024-04-26T04:32:40.428 INFO:tasks.rados.rados.0.smithi178.stdout:145: oid 31 updating version 0 to 18 2024-04-26T04:32:40.428 INFO:tasks.rados.rados.0.smithi178.stdout:145: oid 31 updating version 18 to 19 2024-04-26T04:32:40.428 INFO:tasks.rados.rados.0.smithi178.stdout:145: oid 31 version 19 is already newer than 17 2024-04-26T04:32:40.428 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 31 v 19 (ObjNum 75 snap 7 seq_num 75) dirty exists 2024-04-26T04:32:40.428 INFO:tasks.rados.rados.0.smithi178.stdout:145: left oid 31 (ObjNum 75 snap 7 seq_num 75) 2024-04-26T04:32:40.510 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:40.506+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:40.525 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:40.526 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:32:40.526 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:32:40.526 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:32:40.526 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.526 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 32/30 ECSubWrite(tid=0, reqid=client.4670.0:1, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:40.526 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.202677+0000", 2024-04-26T04:32:40.526 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 15.321885434, 2024-04-26T04:32:40.526 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19503779500000001, 2024-04-26T04:32:40.526 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.526 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.526 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202677+0000", 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202677+0000", 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202678+0000", 2024-04-26T04:32:40.527 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8599999999999998e-07 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202736+0000", 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7933000000000003e-05 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202761+0000", 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4706999999999999e-05 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.528 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202786+0000", 2024-04-26T04:32:40.529 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5528999999999999e-05 2024-04-26T04:32:40.529 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.529 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.529 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.529 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.304898+0000", 2024-04-26T04:32:40.529 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102111946 2024-04-26T04:32:40.529 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.529 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.530 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.530 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.304915+0000", 2024-04-26T04:32:40.530 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6846000000000001e-05 2024-04-26T04:32:40.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.530 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.530 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.530 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397689+0000", 2024-04-26T04:32:40.530 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.092774051999999996 2024-04-26T04:32:40.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.530 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.530 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.530 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397715+0000", 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6196000000000001e-05 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 32/30 ECSubWrite(tid=1, reqid=client.4670.0:2, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.204544+0000", 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 15.320018444, 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19320447099999999, 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.531 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.532 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.532 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.532 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204544+0000", 2024-04-26T04:32:40.532 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.532 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.532 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.532 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.532 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204544+0000", 2024-04-26T04:32:40.532 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.532 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204546+0000", 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.336e-06 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204575+0000", 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9689999999999999e-05 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204597+0000", 2024-04-26T04:32:40.533 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1593000000000001e-05 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.304678+0000", 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10008115400000001 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305492+0000", 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000813666 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.534 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305505+0000", 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3169999999999999e-05 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397718+0000", 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.092213081000000002 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397749+0000", 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0781000000000002e-05 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.535 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.0s5 32/30 ECSubWrite(tid=0, reqid=client.4670.0:22, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.207578+0000", 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 15.316985154999999, 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19009067299999999, 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207578+0000", 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.536 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207578+0000", 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207579+0000", 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6270000000000001e-06 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207691+0000", 2024-04-26T04:32:40.537 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000111382 2024-04-26T04:32:40.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.538 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.538 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.538 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207728+0000", 2024-04-26T04:32:40.538 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6946999999999997e-05 2024-04-26T04:32:40.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.538 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.538 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.538 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207755+0000", 2024-04-26T04:32:40.538 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7472999999999999e-05 2024-04-26T04:32:40.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.538 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.538 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.539 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.297516+0000", 2024-04-26T04:32:40.539 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.089760928000000004 2024-04-26T04:32:40.539 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.539 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.539 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.539 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.297533+0000", 2024-04-26T04:32:40.539 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7249999999999999e-05 2024-04-26T04:32:40.539 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.539 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.539 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.539 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397639+0000", 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100105334 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397668+0000", 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9731999999999999e-05 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.0s5 32/30 ECSubWrite(tid=1, reqid=client.4670.0:23, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.208255+0000", 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 15.316307621, 2024-04-26T04:32:40.540 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.189431346, 2024-04-26T04:32:40.541 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.541 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.541 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.541 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.541 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.541 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.541 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208255+0000", 2024-04-26T04:32:40.541 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.541 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.541 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.542 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208255+0000", 2024-04-26T04:32:40.542 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.542 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.542 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.542 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208257+0000", 2024-04-26T04:32:40.542 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4869999999999999e-06 2024-04-26T04:32:40.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.542 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.542 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.542 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208294+0000", 2024-04-26T04:32:40.542 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7679999999999998e-05 2024-04-26T04:32:40.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.543 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.543 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.543 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208325+0000", 2024-04-26T04:32:40.543 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0533000000000001e-05 2024-04-26T04:32:40.543 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.543 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.543 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.543 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208348+0000", 2024-04-26T04:32:40.543 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2566999999999999e-05 2024-04-26T04:32:40.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.544 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.544 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.544 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.298496+0000", 2024-04-26T04:32:40.544 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.090148454000000003 2024-04-26T04:32:40.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.544 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.544 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.544 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.298518+0000", 2024-04-26T04:32:40.544 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2234000000000001e-05 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397672+0000", 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.099153924000000004 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397687+0000", 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4467000000000001e-05 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.545 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.es5 35/30 ECSubWrite(tid=187, reqid=client.4670.0:233, at_version=35'10, trim_to=0'0, roll_forward_to=34'8))", 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:28.913447+0000", 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 11.611116279000001, 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102676145, 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.913447+0000", 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.546 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.913447+0000", 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.913447+0000", 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6000000000000004e-07 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.913507+0000", 2024-04-26T04:32:40.547 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9873000000000001e-05 2024-04-26T04:32:40.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.548 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.548 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.548 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.913531+0000", 2024-04-26T04:32:40.548 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3615999999999999e-05 2024-04-26T04:32:40.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.548 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.548 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.548 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.013673+0000", 2024-04-26T04:32:40.548 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100142774 2024-04-26T04:32:40.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.548 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.013831+0000", 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015723899999999999 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.013856+0000", 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4986000000000001e-05 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.016058+0000", 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0022022579999999999 2024-04-26T04:32:40.549 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.016123+0000", 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4838999999999994e-05 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.es5 35/30 ECSubWrite(tid=188, reqid=client.4670.0:234, at_version=35'11, trim_to=0'0, roll_forward_to=34'8))", 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:29.013728+0000", 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 11.510835286000001, 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101847233, 2024-04-26T04:32:40.550 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.013728+0000", 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.013728+0000", 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.551 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.013729+0000", 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.263e-06 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.013774+0000", 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4805999999999998e-05 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.013823+0000", 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9137000000000001e-05 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.552 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.553 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.553 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.114002+0000", 2024-04-26T04:32:40.553 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10017888699999999 2024-04-26T04:32:40.553 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.553 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.553 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.553 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.114180+0000", 2024-04-26T04:32:40.553 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017800199999999999 2024-04-26T04:32:40.553 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.114196+0000", 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6750000000000001e-05 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.115498+0000", 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0013015489999999999 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.115575+0000", 2024-04-26T04:32:40.554 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6839000000000001e-05 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.5s6 36/30 ECSubWrite(tid=29, reqid=client.4670.0:266, at_version=36'17, trim_to=0'0, roll_forward_to=35'16))", 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:29.820211+0000", 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.704352338, 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101906232, 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.555 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.820211+0000", 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.820211+0000", 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.820211+0000", 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3700000000000001e-07 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.556 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.820289+0000", 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7861999999999994e-05 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.820322+0000", 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3726999999999997e-05 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.920415+0000", 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100092475 2024-04-26T04:32:40.557 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.920592+0000", 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017676899999999999 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.920606+0000", 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4358999999999999e-05 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.922036+0000", 2024-04-26T04:32:40.558 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001429678 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.922117+0000", 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1025000000000004e-05 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 36/30 ECSubWrite(tid=39, reqid=client.4670.0:247, at_version=36'25, trim_to=0'0, roll_forward_to=35'24))", 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.074094+0000", 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.450468670999999, 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20298102000000001, 2024-04-26T04:32:40.559 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074094+0000", 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074094+0000", 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.560 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074096+0000", 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.854e-06 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074602+0000", 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00050626899999999995 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074642+0000", 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0113e-05 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.561 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174816+0000", 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10017388200000001 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174981+0000", 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016466500000000001 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174995+0000", 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.419e-05 2024-04-26T04:32:40.562 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.563 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.563 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.563 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.277024+0000", 2024-04-26T04:32:40.563 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102028498 2024-04-26T04:32:40.563 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.563 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.563 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.563 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.277075+0000", 2024-04-26T04:32:40.563 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1548999999999997e-05 2024-04-26T04:32:40.563 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.563 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.564 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.564 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.564 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.564 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 36/30 ECSubWrite(tid=40, reqid=client.4670.0:248, at_version=36'26, trim_to=0'0, roll_forward_to=35'24))", 2024-04-26T04:32:40.564 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.174868+0000", 2024-04-26T04:32:40.564 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.34969517, 2024-04-26T04:32:40.564 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102252619, 2024-04-26T04:32:40.564 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.564 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.564 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.564 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.564 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174868+0000", 2024-04-26T04:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174868+0000", 2024-04-26T04:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174869+0000", 2024-04-26T04:32:40.565 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0669999999999999e-06 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.175053+0000", 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00018389799999999999 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.175075+0000", 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2765999999999999e-05 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.175091+0000", 2024-04-26T04:32:40.566 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5829999999999999e-05 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.175832+0000", 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00074106300000000003 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.175845+0000", 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2454e-05 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.567 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.277081+0000", 2024-04-26T04:32:40.568 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101236591 2024-04-26T04:32:40.568 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.568 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.568 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.568 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.277120+0000", 2024-04-26T04:32:40.568 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8949999999999998e-05 2024-04-26T04:32:40.568 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.568 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.568 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.568 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 36/30 ECSubWrite(tid=41, reqid=client.4670.0:249, at_version=36'27, trim_to=0'0, roll_forward_to=35'24))", 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.175118+0000", 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.349445089, 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102835651, 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.175118+0000", 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.569 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.570 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.570 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.570 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.175118+0000", 2024-04-26T04:32:40.570 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.570 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.570 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.570 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.570 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.175118+0000", 2024-04-26T04:32:40.570 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6000000000000003e-07 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.175138+0000", 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9783e-05 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.175150+0000", 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.166e-05 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.571 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.175159+0000", 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8929999999999994e-06 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.276719+0000", 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101560049 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.276737+0000", 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7932999999999999e-05 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.572 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.573 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.277885+0000", 2024-04-26T04:32:40.573 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0011485429999999999 2024-04-26T04:32:40.573 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.573 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.573 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.573 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.277953+0000", 2024-04-26T04:32:40.573 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.813e-05 2024-04-26T04:32:40.573 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.573 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.573 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.574 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.574 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.574 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.2s3 36/30 ECSubWrite(tid=194, reqid=client.4670.0:273, at_version=36'9, trim_to=0'0, roll_forward_to=34'7))", 2024-04-26T04:32:40.574 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.432996+0000", 2024-04-26T04:32:40.574 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.091566454000001, 2024-04-26T04:32:40.574 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102137351, 2024-04-26T04:32:40.574 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.432996+0000", 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.432996+0000", 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.575 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.432997+0000", 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4300000000000002e-07 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.433043+0000", 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6167e-05 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.433068+0000", 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4516e-05 2024-04-26T04:32:40.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.433082+0000", 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4039999999999999e-05 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.533312+0000", 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100229954 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.533326+0000", 2024-04-26T04:32:40.577 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4175999999999999e-05 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.535065+0000", 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0017394649999999999 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.535134+0000", 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.8289999999999998e-05 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 36/30 ECSubWrite(tid=82, reqid=client.4670.0:272, at_version=0'0, trim_to=0'0, roll_forward_to=36'10))", 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.662242+0000", 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 9.8623210999999991, 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101851021, 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.662242+0000", 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.579 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.662242+0000", 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.662243+0000", 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6840000000000001e-06 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.662265+0000", 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1265999999999999e-05 2024-04-26T04:32:40.580 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.662294+0000", 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9753000000000001e-05 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.662310+0000", 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5203e-05 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.762590+0000", 2024-04-26T04:32:40.581 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100279984 2024-04-26T04:32:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.762623+0000", 2024-04-26T04:32:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3439999999999998e-05 2024-04-26T04:32:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.582 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.764001+0000", 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0013780649999999999 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.764093+0000", 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.1626000000000007e-05 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 40/30 ECSubWrite(tid=46, reqid=osd.5.0:46, at_version=40'31, trim_to=0'0, roll_forward_to=39'29))", 2024-04-26T04:32:40.583 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.419586+0000", 2024-04-26T04:32:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.1049764599999996, 2024-04-26T04:32:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10595267, 2024-04-26T04:32:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.584 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419586+0000", 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419586+0000", 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419587+0000", 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5700000000000002e-07 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.585 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419629+0000", 2024-04-26T04:32:40.586 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1628999999999997e-05 2024-04-26T04:32:40.586 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.586 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.586 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.586 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419654+0000", 2024-04-26T04:32:40.586 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5466999999999999e-05 2024-04-26T04:32:40.586 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.586 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.586 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.586 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419670+0000", 2024-04-26T04:32:40.586 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5699999999999999e-05 2024-04-26T04:32:40.586 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419799+0000", 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000129085 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419813+0000", 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4387e-05 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.525472+0000", 2024-04-26T04:32:40.587 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105658988 2024-04-26T04:32:40.588 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.588 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.588 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.588 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.525539+0000", 2024-04-26T04:32:40.588 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6656999999999996e-05 2024-04-26T04:32:40.588 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.588 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.588 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.589 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.589 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.589 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.as5 40/30 ECSubWrite(tid=28, reqid=client.4670.0:330, at_version=40'20, trim_to=0'0, roll_forward_to=38'16))", 2024-04-26T04:32:40.589 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.518282+0000", 2024-04-26T04:32:40.589 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.0062809030000004, 2024-04-26T04:32:40.589 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103325785, 2024-04-26T04:32:40.589 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.589 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.589 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.589 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.589 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.589 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.589 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518282+0000", 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518282+0000", 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518284+0000", 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2869999999999999e-06 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.590 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518328+0000", 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3532000000000003e-05 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518375+0000", 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7194000000000002e-05 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518400+0000", 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5466e-05 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.591 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.592 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.592 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.618615+0000", 2024-04-26T04:32:40.592 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10021411600000001 2024-04-26T04:32:40.592 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.592 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.592 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.592 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.618643+0000", 2024-04-26T04:32:40.592 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7956000000000001e-05 2024-04-26T04:32:40.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.593 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.593 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.593 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.621518+0000", 2024-04-26T04:32:40.593 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002875945 2024-04-26T04:32:40.593 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.593 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.593 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.593 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.621608+0000", 2024-04-26T04:32:40.593 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.9289000000000005e-05 2024-04-26T04:32:40.593 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.593 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.593 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.594 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.594 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.594 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 40/30 ECSubWrite(tid=104, reqid=client.4670.0:321, at_version=40'18, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:32:40.594 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.611130+0000", 2024-04-26T04:32:40.594 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.9134327820000001, 2024-04-26T04:32:40.594 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201625046, 2024-04-26T04:32:40.594 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.594 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.594 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611130+0000", 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611130+0000", 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.595 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611131+0000", 2024-04-26T04:32:40.596 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1300000000000001e-07 2024-04-26T04:32:40.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.596 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.596 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.596 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611749+0000", 2024-04-26T04:32:40.596 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00061894499999999998 2024-04-26T04:32:40.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.596 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.596 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.596 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611779+0000", 2024-04-26T04:32:40.596 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9969999999999999e-05 2024-04-26T04:32:40.596 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.596 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.597 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.597 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611795+0000", 2024-04-26T04:32:40.597 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5443000000000001e-05 2024-04-26T04:32:40.597 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.597 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.597 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.597 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613563+0000", 2024-04-26T04:32:40.597 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001768211 2024-04-26T04:32:40.597 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.597 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.597 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.597 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613590+0000", 2024-04-26T04:32:40.597 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6633000000000001e-05 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812660+0000", 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19907063799999999 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812755+0000", 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4792999999999999e-05 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.598 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 40/30 ECSubWrite(tid=106, reqid=client.4670.0:322, at_version=40'19, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.613168+0000", 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.911394467, 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.199657323, 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613168+0000", 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.599 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.600 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.600 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.600 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613168+0000", 2024-04-26T04:32:40.600 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.600 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.600 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.600 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.600 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613174+0000", 2024-04-26T04:32:40.600 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9569999999999999e-06 2024-04-26T04:32:40.600 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.600 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.600 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.600 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613348+0000", 2024-04-26T04:32:40.601 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000173535 2024-04-26T04:32:40.601 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.602 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.602 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.602 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613377+0000", 2024-04-26T04:32:40.602 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9522999999999998e-05 2024-04-26T04:32:40.602 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.602 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.602 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.602 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613394+0000", 2024-04-26T04:32:40.602 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.628e-05 2024-04-26T04:32:40.602 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.603 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.603 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.603 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.615501+0000", 2024-04-26T04:32:40.603 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021068710000000002 2024-04-26T04:32:40.603 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.603 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.603 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.603 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.615545+0000", 2024-04-26T04:32:40.604 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4406999999999997e-05 2024-04-26T04:32:40.604 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.604 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.604 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.604 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812776+0000", 2024-04-26T04:32:40.604 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19723128400000001 2024-04-26T04:32:40.604 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.604 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.604 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.604 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812826+0000", 2024-04-26T04:32:40.604 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9465999999999997e-05 2024-04-26T04:32:40.604 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.605 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.605 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.605 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.605 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.605 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 40/30 ECSubWrite(tid=107, reqid=client.4670.0:323, at_version=40'20, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:32:40.605 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.613975+0000", 2024-04-26T04:32:40.605 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.9105882469999997, 2024-04-26T04:32:40.605 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.199514147, 2024-04-26T04:32:40.605 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.605 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.605 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613975+0000", 2024-04-26T04:32:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613975+0000", 2024-04-26T04:32:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.606 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613976+0000", 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8029999999999999e-06 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.614012+0000", 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5283000000000001e-05 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.614038+0000", 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5890000000000001e-05 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.607 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.614070+0000", 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1992999999999997e-05 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812143+0000", 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.1980729 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812157+0000", 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.472e-05 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.608 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.813409+0000", 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012518290000000001 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.813489+0000", 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9728999999999995e-05 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.609 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.es5 41/30 ECSubWrite(tid=197, reqid=client.4670.0:349, at_version=41'14, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:35.628948+0000", 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.8956149719999997, 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104618195, 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.628948+0000", 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.610 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.628948+0000", 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.628949+0000", 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2399999999999997e-07 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.629254+0000", 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030504300000000001 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.611 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.629283+0000", 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.885e-05 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.729430+0000", 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100147071 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.729622+0000", 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000192015 2024-04-26T04:32:40.612 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.613 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.613 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.613 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.729640+0000", 2024-04-26T04:32:40.613 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8202999999999998e-05 2024-04-26T04:32:40.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.613 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.613 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.613 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.733492+0000", 2024-04-26T04:32:40.613 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0038524319999999998 2024-04-26T04:32:40.613 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.733566+0000", 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3757000000000006e-05 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 45/30 ECSubWrite(tid=136, reqid=client.4670.0:370, at_version=45'17, trim_to=0'0, roll_forward_to=44'15))", 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:39.405567+0000", 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 1.1189962280000001, 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103082787, 2024-04-26T04:32:40.614 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.615 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.615 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.615 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.615 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.615 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.615 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.405567+0000", 2024-04-26T04:32:40.615 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.615 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.615 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.615 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.615 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.405567+0000", 2024-04-26T04:32:40.616 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.616 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.616 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.616 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.405568+0000", 2024-04-26T04:32:40.616 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1370000000000001e-06 2024-04-26T04:32:40.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.616 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.616 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.616 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.405844+0000", 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00027591999999999997 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.405873+0000", 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9357e-05 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.405890+0000", 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7067000000000001e-05 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.617 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.506122+0000", 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100231508 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.506141+0000", 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8940000000000002e-05 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.508547+0000", 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0024060790000000002 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.618 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.508649+0000", 2024-04-26T04:32:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000102779 2024-04-26T04:32:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 45/30 ECSubWrite(tid=137, reqid=client.4670.0:371, at_version=45'18, trim_to=0'0, roll_forward_to=44'15))", 2024-04-26T04:32:40.619 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:39.609106+0000", 2024-04-26T04:32:40.620 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.91545699899999999, 2024-04-26T04:32:40.620 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102933492, 2024-04-26T04:32:40.620 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:40.620 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:40.620 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:40.620 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:40.620 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.620 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:40.620 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.609106+0000", 2024-04-26T04:32:40.620 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.620 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.620 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.609106+0000", 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.609108+0000", 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9099999999999999e-06 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.609453+0000", 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000344984 2024-04-26T04:32:40.621 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.622 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.622 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:40.622 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.609484+0000", 2024-04-26T04:32:40.622 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0777e-05 2024-04-26T04:32:40.622 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.622 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.622 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:40.622 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.709576+0000", 2024-04-26T04:32:40.622 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100092269 2024-04-26T04:32:40.622 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.623 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.623 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:40.623 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.709738+0000", 2024-04-26T04:32:40.623 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016244499999999999 2024-04-26T04:32:40.623 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.623 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.623 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:40.623 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.709764+0000", 2024-04-26T04:32:40.623 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5357000000000001e-05 2024-04-26T04:32:40.623 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.624 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.624 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:40.624 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.711924+0000", 2024-04-26T04:32:40.624 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002160361 2024-04-26T04:32:40.624 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:40.624 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:40.624 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:40.624 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.712039+0000", 2024-04-26T04:32:40.624 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000115389 2024-04-26T04:32:40.624 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.624 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.624 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.624 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:40.625 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:40.625 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:40.627 INFO:tasks.rados.rados.0.smithi178.stdout:146: finishing write tid 5 to smithi17829308-16 2024-04-26T04:32:40.627 INFO:tasks.rados.rados.0.smithi178.stdout:146: oid 16 updating version 0 to 21 2024-04-26T04:32:40.628 INFO:tasks.rados.rados.0.smithi178.stdout:146: oid 16 updating version 21 to 23 2024-04-26T04:32:40.628 INFO:tasks.rados.rados.0.smithi178.stdout:146: oid 16 version 23 is already newer than 22 2024-04-26T04:32:40.628 INFO:tasks.rados.rados.0.smithi178.stdout:146: oid 16 version 23 is already newer than 20 2024-04-26T04:32:40.628 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 16 v 23 (ObjNum 76 snap 7 seq_num 76) dirty exists 2024-04-26T04:32:40.628 INFO:tasks.rados.rados.0.smithi178.stdout:146: left oid 16 (ObjNum 76 snap 7 seq_num 76) 2024-04-26T04:32:40.628 INFO:tasks.rados.rados.0.smithi178.stdout:145: done (5 left) 2024-04-26T04:32:40.628 INFO:tasks.rados.rados.0.smithi178.stdout:146: done (4 left) 2024-04-26T04:32:40.628 INFO:tasks.rados.rados.0.smithi178.stdout:148: done (3 left) 2024-04-26T04:32:40.629 INFO:tasks.rados.rados.0.smithi178.stdout:149: done (2 left) 2024-04-26T04:32:40.629 INFO:tasks.rados.rados.0.smithi178.stdout:151: done (1 left) 2024-04-26T04:32:40.629 INFO:tasks.rados.rados.0.smithi178.stdout:152: snap_remove snap 5 2024-04-26T04:32:40.629 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:40.622+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:40.629 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T04:32:40.728 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:40.726+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:40.759 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:40.759 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:40.759 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:32:40.759 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:40.773 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T04:32:40.829 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:40.825+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:40.893 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:40.893 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:40.893 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:32:40.893 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:32:40.893 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:40.907 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T04:32:40.930 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:40.926+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:41.025 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:41.025 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:32:41.025 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:32:41.025 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:32:41.025 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.026 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 32/30 ECSubWrite(tid=0, reqid=client.4670.0:4, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:41.026 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.208612+0000", 2024-04-26T04:32:41.026 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 15.816738421, 2024-04-26T04:32:41.026 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101126328, 2024-04-26T04:32:41.026 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.026 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.026 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.026 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.026 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.026 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.026 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208612+0000", 2024-04-26T04:32:41.026 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.027 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.027 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208612+0000", 2024-04-26T04:32:41.027 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.027 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.027 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208612+0000", 2024-04-26T04:32:41.027 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7100000000000001e-07 2024-04-26T04:32:41.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.027 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.027 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208660+0000", 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.8284000000000001e-05 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208668+0000", 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1429999999999998e-06 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208676+0000", 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.61e-06 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.028 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.308804+0000", 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100128461 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.308812+0000", 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2189999999999998e-06 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.309703+0000", 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00089089900000000001 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.029 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.030 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.030 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.309738+0000", 2024-04-26T04:32:41.030 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5540999999999997e-05 2024-04-26T04:32:41.030 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.030 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.030 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.030 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.030 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.030 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 32/30 ECSubWrite(tid=1, reqid=client.4670.0:5, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:41.030 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.212559+0000", 2024-04-26T04:32:41.030 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 15.812791429000001, 2024-04-26T04:32:41.030 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.097396983000000006, 2024-04-26T04:32:41.030 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.031 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.031 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.031 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.031 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.031 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.031 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.212559+0000", 2024-04-26T04:32:41.031 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.031 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.031 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.031 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.031 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.212559+0000", 2024-04-26T04:32:41.031 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.031 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.212560+0000", 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7630000000000001e-06 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.212592+0000", 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1409999999999999e-05 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.212620+0000", 2024-04-26T04:32:41.032 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7793e-05 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.212636+0000", 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5866999999999999e-05 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.309158+0000", 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.096522601 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.033 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.309166+0000", 2024-04-26T04:32:41.034 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6539999999999999e-06 2024-04-26T04:32:41.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.034 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.034 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.034 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.309924+0000", 2024-04-26T04:32:41.034 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00075805899999999995 2024-04-26T04:32:41.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.034 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.034 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.309956+0000", 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1835999999999998e-05 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 33/30 ECSubWrite(tid=5, reqid=client.4670.0:38, at_version=33'6, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.444642+0000", 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 15.580707867999999, 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0035635469999999998, 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.035 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.036 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.036 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.036 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.444642+0000", 2024-04-26T04:32:41.036 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.036 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.036 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.036 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.444642+0000", 2024-04-26T04:32:41.036 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.036 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.036 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.036 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.444643+0000", 2024-04-26T04:32:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.7999999999999993e-07 2024-04-26T04:32:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.444673+0000", 2024-04-26T04:32:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.934e-05 2024-04-26T04:32:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.444693+0000", 2024-04-26T04:32:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0888999999999999e-05 2024-04-26T04:32:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.037 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.444711+0000", 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7833999999999999e-05 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.445630+0000", 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00091885399999999998 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.445639+0000", 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.9509999999999995e-06 2024-04-26T04:32:41.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.448170+0000", 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0025307060000000002 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.448206+0000", 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5992999999999999e-05 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.040 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 35/30 ECSubWrite(tid=19, reqid=client.4670.0:189, at_version=0'0, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T04:32:41.040 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:28.969326+0000", 2024-04-26T04:32:41.040 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 12.056024361, 2024-04-26T04:32:41.040 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101585699, 2024-04-26T04:32:41.040 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.040 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.040 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.040 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.040 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.040 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.040 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.969326+0000", 2024-04-26T04:32:41.040 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.040 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.969326+0000", 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.969328+0000", 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8470000000000001e-06 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.969348+0000", 2024-04-26T04:32:41.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0699999999999998e-05 2024-04-26T04:32:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.969379+0000", 2024-04-26T04:32:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0772999999999998e-05 2024-04-26T04:32:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.069523+0000", 2024-04-26T04:32:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014342399999999 2024-04-26T04:32:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.042 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.043 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.069732+0000", 2024-04-26T04:32:41.043 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00020916199999999999 2024-04-26T04:32:41.043 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.043 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.043 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.043 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.069789+0000", 2024-04-26T04:32:41.043 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7216000000000002e-05 2024-04-26T04:32:41.043 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.043 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.043 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.043 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.070835+0000", 2024-04-26T04:32:41.043 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001046324 2024-04-26T04:32:41.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.044 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.044 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.044 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.070911+0000", 2024-04-26T04:32:41.044 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6253000000000003e-05 2024-04-26T04:32:41.044 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.044 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.044 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.045 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 35/30 ECSubWrite(tid=61, reqid=client.4670.0:211, at_version=0'0, trim_to=0'0, roll_forward_to=35'17))", 2024-04-26T04:32:41.045 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:29.330344+0000", 2024-04-26T04:32:41.045 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 11.695006415, 2024-04-26T04:32:41.045 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105013624, 2024-04-26T04:32:41.045 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.045 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.045 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.045 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.045 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.046 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330344+0000", 2024-04-26T04:32:41.046 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.046 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.046 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.046 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330344+0000", 2024-04-26T04:32:41.046 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.046 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.046 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.046 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330346+0000", 2024-04-26T04:32:41.046 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1399999999999998e-06 2024-04-26T04:32:41.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.047 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.047 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.047 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330371+0000", 2024-04-26T04:32:41.047 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5320000000000002e-05 2024-04-26T04:32:41.047 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.047 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.047 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.047 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330403+0000", 2024-04-26T04:32:41.047 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2305999999999999e-05 2024-04-26T04:32:41.047 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.047 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.047 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.047 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330423+0000", 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9179000000000001e-05 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.433931+0000", 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103508818 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.433962+0000", 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0576e-05 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.048 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.049 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.435260+0000", 2024-04-26T04:32:41.049 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001297819 2024-04-26T04:32:41.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.049 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.049 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.049 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.435357+0000", 2024-04-26T04:32:41.049 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.7466000000000005e-05 2024-04-26T04:32:41.049 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.049 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.049 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.049 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.049 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=63, reqid=client.4670.0:236, at_version=0'0, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:41.050 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:29.738155+0000", 2024-04-26T04:32:41.050 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 11.287194783, 2024-04-26T04:32:41.050 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10169805799999999, 2024-04-26T04:32:41.050 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.050 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.050 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.050 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.050 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.050 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.050 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.738155+0000", 2024-04-26T04:32:41.050 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.738155+0000", 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.738157+0000", 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6729999999999999e-06 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.738178+0000", 2024-04-26T04:32:41.051 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1053e-05 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.738208+0000", 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0017000000000001e-05 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.738228+0000", 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9516000000000001e-05 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.838511+0000", 2024-04-26T04:32:41.052 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10028327400000001 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.838535+0000", 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4326000000000001e-05 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.839782+0000", 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001247272 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.053 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.839853+0000", 2024-04-26T04:32:41.054 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0926999999999994e-05 2024-04-26T04:32:41.054 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.054 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.054 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.054 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.054 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.054 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=68, reqid=client.4670.0:255, at_version=36'23, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:41.054 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.152243+0000", 2024-04-26T04:32:41.054 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.873106802000001, 2024-04-26T04:32:41.054 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10112824300000001, 2024-04-26T04:32:41.054 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.054 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.054 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152243+0000", 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152243+0000", 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.055 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152245+0000", 2024-04-26T04:32:41.056 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9e-06 2024-04-26T04:32:41.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.056 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.056 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.056 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152462+0000", 2024-04-26T04:32:41.056 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000217245 2024-04-26T04:32:41.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.056 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.056 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.056 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152493+0000", 2024-04-26T04:32:41.056 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0703000000000001e-05 2024-04-26T04:32:41.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.057 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.057 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152519+0000", 2024-04-26T04:32:41.057 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5914e-05 2024-04-26T04:32:41.057 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.057 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.057 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152690+0000", 2024-04-26T04:32:41.057 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000170481 2024-04-26T04:32:41.057 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.057 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.057 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152704+0000", 2024-04-26T04:32:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4256999999999999e-05 2024-04-26T04:32:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.253310+0000", 2024-04-26T04:32:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10060620100000001 2024-04-26T04:32:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.253372+0000", 2024-04-26T04:32:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.1542000000000001e-05 2024-04-26T04:32:41.058 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.059 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.059 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.059 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.059 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.059 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=69, reqid=client.4670.0:256, at_version=36'24, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:41.059 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.152566+0000", 2024-04-26T04:32:41.059 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.872783994000001, 2024-04-26T04:32:41.059 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100858378, 2024-04-26T04:32:41.059 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.059 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.059 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.059 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.060 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.060 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.060 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152566+0000", 2024-04-26T04:32:41.060 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.060 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.060 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.060 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.060 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152566+0000", 2024-04-26T04:32:41.060 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.060 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.060 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.060 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152567+0000", 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.136e-06 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152692+0000", 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000125129 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152726+0000", 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3423000000000002e-05 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.061 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152750+0000", 2024-04-26T04:32:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3844000000000001e-05 2024-04-26T04:32:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.153436+0000", 2024-04-26T04:32:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00068632999999999997 2024-04-26T04:32:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.153451+0000", 2024-04-26T04:32:41.062 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4780000000000001e-05 2024-04-26T04:32:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.253391+0000", 2024-04-26T04:32:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.099940213 2024-04-26T04:32:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.253425+0000", 2024-04-26T04:32:41.063 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3522999999999997e-05 2024-04-26T04:32:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=70, reqid=client.4670.0:257, at_version=36'25, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.152786+0000", 2024-04-26T04:32:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.872564483, 2024-04-26T04:32:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101303855, 2024-04-26T04:32:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.064 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152786+0000", 2024-04-26T04:32:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152786+0000", 2024-04-26T04:32:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.065 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152787+0000", 2024-04-26T04:32:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0669999999999999e-06 2024-04-26T04:32:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152820+0000", 2024-04-26T04:32:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2793000000000003e-05 2024-04-26T04:32:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.066 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152841+0000", 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1503000000000001e-05 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.152862+0000", 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0557e-05 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.252993+0000", 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10013169800000001 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.067 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.253008+0000", 2024-04-26T04:32:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4323e-05 2024-04-26T04:32:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.254023+0000", 2024-04-26T04:32:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001015555 2024-04-26T04:32:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.068 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.254090+0000", 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6359000000000006e-05 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpRead(3.2s1 36/30 ECSubRead(tid=192, to_read={3:4497087f:::smithi17829308-22:head=0,90112,0}, subchunks={3:4497087f:::smithi17829308-22:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.231971+0000", 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.793378979, 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.1004024, 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.069 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.231971+0000", 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.231971+0000", 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.231972+0000", 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5000000000000002e-07 2024-04-26T04:32:41.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.231983+0000", 2024-04-26T04:32:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1657e-05 2024-04-26T04:32:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.232029+0000", 2024-04-26T04:32:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5945999999999997e-05 2024-04-26T04:32:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.071 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.232053+0000", 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3413000000000001e-05 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.332195+0000", 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014182100000001 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.332374+0000", 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000178913 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.072 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=73, reqid=client.4670.0:259, at_version=36'27, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.257763+0000", 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.76758673, 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103026436, 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.257763+0000", 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.257763+0000", 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.257766+0000", 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.216e-06 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258080+0000", 2024-04-26T04:32:41.074 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00031390699999999999 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258110+0000", 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0497e-05 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258127+0000", 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6663e-05 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.075 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.358401+0000", 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10027383099999999 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.358415+0000", 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4207e-05 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.360708+0000", 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0022931499999999999 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.076 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.360790+0000", 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1965000000000006e-05 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=74, reqid=client.4670.0:260, at_version=36'28, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.258165+0000", 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.767185304, 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10329530100000001, 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.077 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258165+0000", 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258165+0000", 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.078 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258166+0000", 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8299999999999995e-07 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258269+0000", 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000103069 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258288+0000", 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8652999999999999e-05 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.079 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258299+0000", 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1243999999999999e-05 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.359776+0000", 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10147722000000001 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.359798+0000", 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2476999999999998e-05 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.080 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.361395+0000", 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0015963660000000001 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.361460+0000", 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5289000000000005e-05 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.2s1 36/30 ECSubWrite(tid=194, reqid=client.4670.0:273, at_version=36'9, trim_to=0'0, roll_forward_to=34'7))", 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.432974+0000", 2024-04-26T04:32:41.081 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.592376005, 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10231934299999999, 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.432974+0000", 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.432974+0000", 2024-04-26T04:32:41.082 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.432975+0000", 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8400000000000003e-07 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.433021+0000", 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6053000000000001e-05 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.083 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.433041+0000", 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.982e-05 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.433058+0000", 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7226000000000001e-05 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.533310+0000", 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100251811 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.084 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.533324+0000", 2024-04-26T04:32:41.085 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4545999999999999e-05 2024-04-26T04:32:41.085 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.085 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.085 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.085 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.535227+0000", 2024-04-26T04:32:41.085 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0019027600000000001 2024-04-26T04:32:41.085 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.085 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.085 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.085 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.535293+0000", 2024-04-26T04:32:41.085 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6242999999999996e-05 2024-04-26T04:32:41.085 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.085 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.086 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.086 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.086 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.086 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 36/30 ECSubWrite(tid=36, reqid=client.4670.0:281, at_version=0'0, trim_to=0'0, roll_forward_to=36'18))", 2024-04-26T04:32:41.086 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:31.282198+0000", 2024-04-26T04:32:41.086 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 9.7431520860000003, 2024-04-26T04:32:41.086 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101473652, 2024-04-26T04:32:41.086 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.086 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.086 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.086 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.086 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.086 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.282198+0000", 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.282198+0000", 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.282199+0000", 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7999999999999995e-07 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.087 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.088 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.088 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.282222+0000", 2024-04-26T04:32:41.088 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.304e-05 2024-04-26T04:32:41.088 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.088 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.088 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.088 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.282254+0000", 2024-04-26T04:32:41.088 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2592999999999998e-05 2024-04-26T04:32:41.088 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.088 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.088 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.088 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.382396+0000", 2024-04-26T04:32:41.088 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014199 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.382664+0000", 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00026781400000000001 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.382685+0000", 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0469999999999999e-05 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.089 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.383598+0000", 2024-04-26T04:32:41.090 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00091384500000000004 2024-04-26T04:32:41.090 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.090 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.090 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.090 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.383672+0000", 2024-04-26T04:32:41.090 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3219999999999994e-05 2024-04-26T04:32:41.090 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.090 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.090 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.090 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.090 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.090 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 40/30 ECSubWrite(tid=30, reqid=osd.7.0:30, at_version=40'17, trim_to=0'0, roll_forward_to=39'15))", 2024-04-26T04:32:41.090 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.518234+0000", 2024-04-26T04:32:41.091 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.5071162869999997, 2024-04-26T04:32:41.091 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.099397890000000003, 2024-04-26T04:32:41.091 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.091 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.091 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.091 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.091 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.091 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.091 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518234+0000", 2024-04-26T04:32:41.091 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.091 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.091 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.091 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518234+0000", 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518236+0000", 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8169999999999999e-06 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518281+0000", 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5806000000000002e-05 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.092 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.093 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.093 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518320+0000", 2024-04-26T04:32:41.093 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8556000000000002e-05 2024-04-26T04:32:41.093 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.093 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.093 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.093 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518337+0000", 2024-04-26T04:32:41.093 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6674e-05 2024-04-26T04:32:41.093 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.093 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.093 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.093 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.616114+0000", 2024-04-26T04:32:41.093 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.097777451000000001 2024-04-26T04:32:41.094 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.094 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.094 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.094 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.616143+0000", 2024-04-26T04:32:41.094 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9292e-05 2024-04-26T04:32:41.094 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.094 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.094 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.094 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.617566+0000", 2024-04-26T04:32:41.094 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001422268 2024-04-26T04:32:41.094 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.094 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.094 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.095 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.617632+0000", 2024-04-26T04:32:41.095 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6025999999999994e-05 2024-04-26T04:32:41.095 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.095 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.095 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.095 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.095 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.095 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 40/30 ECSubWrite(tid=105, reqid=client.4670.0:331, at_version=40'34, trim_to=0'0, roll_forward_to=38'33))", 2024-04-26T04:32:41.095 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.612294+0000", 2024-04-26T04:32:41.095 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.4130557130000003, 2024-04-26T04:32:41.095 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101661354, 2024-04-26T04:32:41.095 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.095 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.096 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.096 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.096 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.096 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.096 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.612294+0000", 2024-04-26T04:32:41.096 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.096 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.096 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.096 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.096 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.612294+0000", 2024-04-26T04:32:41.096 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.096 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.096 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.097 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.097 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.612295+0000", 2024-04-26T04:32:41.097 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.03e-07 2024-04-26T04:32:41.097 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.097 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.097 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.097 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.612323+0000", 2024-04-26T04:32:41.097 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7767e-05 2024-04-26T04:32:41.097 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.097 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.097 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.097 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.612348+0000", 2024-04-26T04:32:41.097 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5256e-05 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.712506+0000", 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100157774 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.712718+0000", 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00021275800000000001 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.098 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.712740+0000", 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1137000000000001e-05 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.713874+0000", 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0011344230000000001 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.713956+0000", 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1835999999999994e-05 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.099 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 40/30 ECSubWrite(tid=32, reqid=client.4670.0:332, at_version=40'19, trim_to=0'0, roll_forward_to=40'17))", 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.920408+0000", 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.1049424190000003, 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10273331099999999, 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920408+0000", 2024-04-26T04:32:41.100 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920408+0000", 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920410+0000", 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.813e-06 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.101 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920465+0000", 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5549e-05 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920503+0000", 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7580000000000003e-05 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.020672+0000", 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10016948000000001 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.102 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.020836+0000", 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000164036 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.020861+0000", 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4349999999999999e-05 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023056+0000", 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021951800000000001 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.103 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023141+0000", 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5322999999999996e-05 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 41/30 ECSubWrite(tid=44, reqid=client.4670.0:346, at_version=41'26, trim_to=0'0, roll_forward_to=38'24))", 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.2370622469999999, 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103457721, 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.104 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.105 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4600000000000005e-07 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788918+0000", 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00062948799999999999 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788943+0000", 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4706e-05 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.106 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.889107+0000", 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100164644 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.889285+0000", 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017809600000000001 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.889311+0000", 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5233e-05 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.107 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.108 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.891664+0000", 2024-04-26T04:32:41.108 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0023538460000000001 2024-04-26T04:32:41.108 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.108 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.108 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.108 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.891746+0000", 2024-04-26T04:32:41.108 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1162000000000007e-05 2024-04-26T04:32:41.108 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.108 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.108 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.109 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.109 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.109 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpRead(3.ds3 45/30 ECSubRead(tid=38, to_read={3:b7d7af40:::smithi17829308-48:head=0,200704,0}, subchunks={3:b7d7af40:::smithi17829308-48:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:41.109 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:39.701595+0000", 2024-04-26T04:32:41.109 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 1.323754852, 2024-04-26T04:32:41.109 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101901167, 2024-04-26T04:32:41.109 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.109 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.109 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.109 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.109 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.109 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.109 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.701595+0000", 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.701595+0000", 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.701598+0000", 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6129999999999998e-06 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.110 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.111 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.701622+0000", 2024-04-26T04:32:41.111 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4562999999999998e-05 2024-04-26T04:32:41.111 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.111 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.111 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.111 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.701662+0000", 2024-04-26T04:32:41.111 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9218999999999998e-05 2024-04-26T04:32:41.111 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.111 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.111 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.111 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.701687+0000", 2024-04-26T04:32:41.111 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5527000000000001e-05 2024-04-26T04:32:41.111 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.803375+0000", 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10168782699999999 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.803496+0000", 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000121418 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.112 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpRead(3.ds3 45/30 ECSubRead(tid=39, to_read={3:b7d7af40:::smithi17829308-48:head=0,200704,0}, subchunks={3:b7d7af40:::smithi17829308-48:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:39.702108+0000", 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 1.3232419010000001, 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101803459, 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.702108+0000", 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.113 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.702108+0000", 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.702109+0000", 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1999999999999999e-06 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.702123+0000", 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.359e-05 2024-04-26T04:32:41.114 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.702147+0000", 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3609999999999999e-05 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.702162+0000", 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5440000000000001e-05 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.803620+0000", 2024-04-26T04:32:41.115 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10145747099999999 2024-04-26T04:32:41.116 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.116 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.116 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.116 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.803912+0000", 2024-04-26T04:32:41.116 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000292148 2024-04-26T04:32:41.116 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.116 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.116 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.116 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.116 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.116 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:41.118 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T04:32:41.119 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:41.114+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:41.158 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-04-26T04:32:41.158 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.3 to 1 (10000) 2024-04-26T04:32:41.165 INFO:tasks.rados.rados.0.smithi178.stdout:152: done (1 left) 2024-04-26T04:32:41.165 INFO:tasks.rados.rados.0.smithi178.stdout:153: snap_create 2024-04-26T04:32:41.165 INFO:tasks.rados.rados.0.smithi178.stdout:147: finishing copy_from to smithi17829308-10 2024-04-26T04:32:41.165 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 10 v 2 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-26T04:32:41.184 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-26T04:32:41.218 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:41.216+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:41.249 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:41.249 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:32:41.250 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.250 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 47/30 ECSubWrite(tid=55, reqid=osd.5.0:55, at_version=47'43, trim_to=0'0, roll_forward_to=44'41))", 2024-04-26T04:32:41.250 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:41.211945+0000", 2024-04-26T04:32:41.250 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.039307198000000002, 2024-04-26T04:32:41.250 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.039350433999999997, 2024-04-26T04:32:41.250 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.250 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.250 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:41.250 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.250 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.250 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.250 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.211945+0000", 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.211945+0000", 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.211946+0000", 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6699999999999999e-07 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.211980+0000", 2024-04-26T04:32:41.251 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4139e-05 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.212003+0000", 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3527e-05 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.212043+0000", 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0012999999999998e-05 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.252 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.253 INFO:teuthology.orchestra.run.smithi137.stdout: ], 2024-04-26T04:32:41.253 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 1 2024-04-26T04:32:41.253 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:41.264 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T04:32:41.350 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:41.348+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:41.422 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:41.421+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:41.436 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:41.437 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:41.437 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:32:41.437 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:32:41.437 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:41.450 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T04:32:41.523 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:41.522+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:41.604 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T04:32:41.614 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:41.615 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:32:41.615 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:32:41.615 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:32:41.615 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.615 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:22 3.0s0 3:0061323f:::smithi17829308-8:head [append 0~131072,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:41.615 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.206381+0000", 2024-04-26T04:32:41.615 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.407015288, 2024-04-26T04:32:41.615 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30553245699999998, 2024-04-26T04:32:41.615 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.615 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.616 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:41.616 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:32:41.616 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:32:41.616 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:41.616 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 22 2024-04-26T04:32:41.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.616 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.616 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.616 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.616 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206381+0000", 2024-04-26T04:32:41.616 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.616 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.616 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.617 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.617 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206381+0000", 2024-04-26T04:32:41.617 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.617 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.617 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.617 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206385+0000", 2024-04-26T04:32:41.617 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3299999999999999e-06 2024-04-26T04:32:41.617 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.617 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.617 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.617 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206772+0000", 2024-04-26T04:32:41.618 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000387673 2024-04-26T04:32:41.618 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.618 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.618 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.618 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206797+0000", 2024-04-26T04:32:41.618 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4156e-05 2024-04-26T04:32:41.618 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.618 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.618 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.618 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206811+0000", 2024-04-26T04:32:41.618 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4087000000000001e-05 2024-04-26T04:32:41.618 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.618 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.619 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.619 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206881+0000", 2024-04-26T04:32:41.619 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9948000000000002e-05 2024-04-26T04:32:41.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.619 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.619 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:41.619 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206974+0000", 2024-04-26T04:32:41.619 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.3047000000000005e-05 2024-04-26T04:32:41.619 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.619 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.619 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.619 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207449+0000", 2024-04-26T04:32:41.619 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00047526900000000001 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208764+0000", 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001315395 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511723+0000", 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.302958377 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511914+0000", 2024-04-26T04:32:41.620 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000191175 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:23 3.0s0 3:0061323f:::smithi17829308-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.206854+0000", 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.406542772000002, 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.407632722, 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:32:41.621 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:41.622 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 23 2024-04-26T04:32:41.622 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.622 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.622 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.622 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.622 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206854+0000", 2024-04-26T04:32:41.622 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.622 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.622 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.622 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.622 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206854+0000", 2024-04-26T04:32:41.622 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.623 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.623 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.623 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.623 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206855+0000", 2024-04-26T04:32:41.623 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.133e-06 2024-04-26T04:32:41.623 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.623 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.623 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.623 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206875+0000", 2024-04-26T04:32:41.623 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9945999999999998e-05 2024-04-26T04:32:41.624 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.624 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.624 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.624 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206889+0000", 2024-04-26T04:32:41.624 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3997e-05 2024-04-26T04:32:41.624 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.625 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.625 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.625 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206899+0000", 2024-04-26T04:32:41.625 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8490000000000001e-06 2024-04-26T04:32:41.625 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.625 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.625 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.625 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207769+0000", 2024-04-26T04:32:41.625 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00087023199999999997 2024-04-26T04:32:41.625 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.625 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.626 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:41.626 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207895+0000", 2024-04-26T04:32:41.626 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012591300000000001 2024-04-26T04:32:41.626 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.626 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.626 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.626 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208201+0000", 2024-04-26T04:32:41.626 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030561 2024-04-26T04:32:41.626 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.626 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.626 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.626 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.209436+0000", 2024-04-26T04:32:41.626 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012354969999999999 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.614327+0000", 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.40489055000000002 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.614487+0000", 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015999499999999999 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.627 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:24 3.0s0 3:0061323f:::smithi17829308-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.206918+0000", 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.406478979999999, 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.612165556, 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 24 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.628 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.629 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206918+0000", 2024-04-26T04:32:41.629 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.629 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.630 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.630 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.630 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206918+0000", 2024-04-26T04:32:41.630 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.630 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.630 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.630 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.630 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206919+0000", 2024-04-26T04:32:41.630 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5000000000000001e-07 2024-04-26T04:32:41.630 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.631 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.631 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.631 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206923+0000", 2024-04-26T04:32:41.631 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0330000000000002e-06 2024-04-26T04:32:41.631 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.631 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.631 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.631 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206934+0000", 2024-04-26T04:32:41.631 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0866999999999999e-05 2024-04-26T04:32:41.631 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.631 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.631 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.631 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206946+0000", 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2947e-05 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208374+0000", 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014272930000000001 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208613+0000", 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00023916200000000001 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.632 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.633 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.615561+0000", 2024-04-26T04:32:41.633 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.40694828199999999 2024-04-26T04:32:41.633 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.633 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.633 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:41.633 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.615825+0000", 2024-04-26T04:32:41.633 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00026369700000000002 2024-04-26T04:32:41.633 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.633 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.633 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.634 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.819083+0000", 2024-04-26T04:32:41.634 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20325832499999999 2024-04-26T04:32:41.634 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.634 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.634 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.634 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.634 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.634 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.as4 32/30 ECSubWrite(tid=0, reqid=client.4670.0:40, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:41.634 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.218637+0000", 2024-04-26T04:32:41.634 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.394759224000001, 2024-04-26T04:32:41.634 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10236036, 2024-04-26T04:32:41.634 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.634 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.635 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.635 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.635 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.635 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.635 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.218637+0000", 2024-04-26T04:32:41.635 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.636 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.636 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.636 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.636 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.218637+0000", 2024-04-26T04:32:41.636 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.636 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.636 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.636 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.636 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.218639+0000", 2024-04-26T04:32:41.636 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9700000000000002e-06 2024-04-26T04:32:41.636 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.218900+0000", 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00026033499999999999 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.218927+0000", 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7708999999999998e-05 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.319063+0000", 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10013590999999999 2024-04-26T04:32:41.637 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.638 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.638 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.638 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.319278+0000", 2024-04-26T04:32:41.638 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000214155 2024-04-26T04:32:41.638 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.638 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.638 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.638 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.319306+0000", 2024-04-26T04:32:41.638 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8039999999999999e-05 2024-04-26T04:32:41.638 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.638 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.638 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.638 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.320919+0000", 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016134019999999999 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.320998+0000", 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8838999999999995e-05 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=0, reqid=client.4670.0:1, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:32:41.639 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.310549966, 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.208159126, 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:32:41.640 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8299999999999998e-07 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302898+0000", 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1421000000000001e-05 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.641 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302912+0000", 2024-04-26T04:32:41.642 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4075e-05 2024-04-26T04:32:41.642 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.642 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.642 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.642 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302928+0000", 2024-04-26T04:32:41.642 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5333999999999999e-05 2024-04-26T04:32:41.642 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.642 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.642 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.642 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403201+0000", 2024-04-26T04:32:41.642 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100273162 2024-04-26T04:32:41.642 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.642 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.643 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.643 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403219+0000", 2024-04-26T04:32:41.643 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7526000000000001e-05 2024-04-26T04:32:41.643 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.643 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.643 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.643 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.510967+0000", 2024-04-26T04:32:41.643 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107748329 2024-04-26T04:32:41.643 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.643 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.643 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.643 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511006+0000", 2024-04-26T04:32:41.643 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8995999999999997e-05 2024-04-26T04:32:41.644 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.644 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.644 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.644 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.644 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.644 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=1, reqid=client.4670.0:2, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:41.644 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:32:41.644 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.310446889000001, 2024-04-26T04:32:41.644 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.208096686, 2024-04-26T04:32:41.644 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.644 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.645 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:32:41.646 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.82e-07 2024-04-26T04:32:41.646 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.646 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.646 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.646 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302987+0000", 2024-04-26T04:32:41.646 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6507999999999997e-05 2024-04-26T04:32:41.646 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.646 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.647 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.647 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302996+0000", 2024-04-26T04:32:41.647 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4900000000000006e-06 2024-04-26T04:32:41.647 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.647 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.647 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.647 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303002+0000", 2024-04-26T04:32:41.647 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3800000000000002e-06 2024-04-26T04:32:41.647 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.647 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.648 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.648 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403906+0000", 2024-04-26T04:32:41.648 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100904445 2024-04-26T04:32:41.648 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.648 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.648 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.648 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403919+0000", 2024-04-26T04:32:41.648 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.259e-05 2024-04-26T04:32:41.648 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.648 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.648 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.648 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511012+0000", 2024-04-26T04:32:41.648 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107093034 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511046+0000", 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4756999999999999e-05 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=2, reqid=client.4670.0:13, at_version=32'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.303011+0000", 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.310385425, 2024-04-26T04:32:41.649 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20833385600000001, 2024-04-26T04:32:41.650 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.650 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.650 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.650 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.650 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.650 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.650 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303011+0000", 2024-04-26T04:32:41.650 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.650 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.650 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.650 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.650 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303011+0000", 2024-04-26T04:32:41.650 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303012+0000", 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4700000000000002e-07 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303033+0000", 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1123000000000001e-05 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303040+0000", 2024-04-26T04:32:41.651 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5340000000000002e-06 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303044+0000", 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1620000000000001e-06 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.404280+0000", 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101235978 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.652 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.404303+0000", 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2446000000000002e-05 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511267+0000", 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.1069645 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511345+0000", 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7766e-05 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.653 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.654 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.654 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 32/30 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-04-26T04:32:41.654 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.305887+0000", 2024-04-26T04:32:41.654 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.307509867, 2024-04-26T04:32:41.654 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20516263700000001, 2024-04-26T04:32:41.654 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.654 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.654 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.654 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.654 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305887+0000", 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305887+0000", 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305888+0000", 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.108e-06 2024-04-26T04:32:41.655 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.656 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.656 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.656 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305895+0000", 2024-04-26T04:32:41.656 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2099999999999996e-06 2024-04-26T04:32:41.656 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.656 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.656 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.656 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305914+0000", 2024-04-26T04:32:41.656 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9164000000000002e-05 2024-04-26T04:32:41.656 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.656 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.657 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.657 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305925+0000", 2024-04-26T04:32:41.657 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1156e-05 2024-04-26T04:32:41.657 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.657 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.657 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.657 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.510931+0000", 2024-04-26T04:32:41.657 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20500514 2024-04-26T04:32:41.658 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.658 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.658 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.658 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511049+0000", 2024-04-26T04:32:41.658 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00011885899999999999 2024-04-26T04:32:41.658 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.658 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.658 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.658 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.658 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.658 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 32/30 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-04-26T04:32:41.658 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.306725+0000", 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.306671501, 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20445080600000001, 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306725+0000", 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306725+0000", 2024-04-26T04:32:41.659 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306726+0000", 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2500000000000004e-07 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306731+0000", 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9250000000000003e-06 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306745+0000", 2024-04-26T04:32:41.660 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4367e-05 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306753+0000", 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9699999999999999e-06 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511144+0000", 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20439093 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.661 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511176+0000", 2024-04-26T04:32:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1689000000000001e-05 2024-04-26T04:32:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs2 32/30 ECSubWrite(tid=1, reqid=client.4670.0:8, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.344461+0000", 2024-04-26T04:32:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.268935911, 2024-04-26T04:32:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10304110700000001, 2024-04-26T04:32:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.662 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.344461+0000", 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.344461+0000", 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.663 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.344461+0000", 2024-04-26T04:32:41.664 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3599999999999999e-07 2024-04-26T04:32:41.664 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.664 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.664 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.664 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.344496+0000", 2024-04-26T04:32:41.664 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5179999999999999e-05 2024-04-26T04:32:41.664 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.664 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.665 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.665 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.344521+0000", 2024-04-26T04:32:41.665 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4766e-05 2024-04-26T04:32:41.665 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.665 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.665 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.665 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.444630+0000", 2024-04-26T04:32:41.665 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100108871 2024-04-26T04:32:41.665 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.665 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.666 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.666 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.444733+0000", 2024-04-26T04:32:41.666 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000103014 2024-04-26T04:32:41.666 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.667 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.667 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.667 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.444745+0000", 2024-04-26T04:32:41.667 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.204e-05 2024-04-26T04:32:41.667 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.667 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.667 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.668 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.447426+0000", 2024-04-26T04:32:41.668 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0026813309999999999 2024-04-26T04:32:41.668 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.668 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.668 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.668 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.447502+0000", 2024-04-26T04:32:41.668 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5469000000000005e-05 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 33/30 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.397734+0000", 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.215662436999999, 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.113617151, 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.669 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.670 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397734+0000", 2024-04-26T04:32:41.670 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.670 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.670 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.670 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.670 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397734+0000", 2024-04-26T04:32:41.670 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.670 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.670 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.670 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.670 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397736+0000", 2024-04-26T04:32:41.670 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.933e-06 2024-04-26T04:32:41.670 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397748+0000", 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1433999999999999e-05 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397769+0000", 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1115999999999999e-05 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397794+0000", 2024-04-26T04:32:41.671 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5426999999999999e-05 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511242+0000", 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.11344815499999999 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511351+0000", 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010908600000000001 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.672 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 33/30 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.397821+0000", 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.215576123000002, 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.11354435, 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397821+0000", 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.673 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.674 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.674 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397821+0000", 2024-04-26T04:32:41.674 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.674 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.674 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.674 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.674 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397821+0000", 2024-04-26T04:32:41.674 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.44e-07 2024-04-26T04:32:41.674 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.674 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.674 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.674 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397825+0000", 2024-04-26T04:32:41.674 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8039999999999999e-06 2024-04-26T04:32:41.675 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.675 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.675 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.675 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397836+0000", 2024-04-26T04:32:41.675 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1292e-05 2024-04-26T04:32:41.675 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.675 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.675 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.675 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397844+0000", 2024-04-26T04:32:41.675 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.43e-06 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511322+0000", 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.11347817 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511365+0000", 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3109999999999999e-05 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.676 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.677 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=3, reqid=client.4670.0:14, at_version=32'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:41.677 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.403652+0000", 2024-04-26T04:32:41.677 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.209744235999999, 2024-04-26T04:32:41.677 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.109359838, 2024-04-26T04:32:41.677 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.677 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.677 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.677 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.677 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.677 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.677 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403652+0000", 2024-04-26T04:32:41.677 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.677 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403652+0000", 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403653+0000", 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8099999999999999e-07 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403676+0000", 2024-04-26T04:32:41.678 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3708999999999999e-05 2024-04-26T04:32:41.679 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.679 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.679 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.679 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403695+0000", 2024-04-26T04:32:41.679 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8754e-05 2024-04-26T04:32:41.679 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.679 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.679 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.679 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403710+0000", 2024-04-26T04:32:41.679 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4577e-05 2024-04-26T04:32:41.679 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.679 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.680 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.680 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511469+0000", 2024-04-26T04:32:41.680 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107758928 2024-04-26T04:32:41.680 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.680 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.680 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.680 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511490+0000", 2024-04-26T04:32:41.680 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1036999999999999e-05 2024-04-26T04:32:41.680 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.681 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.681 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.681 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.512951+0000", 2024-04-26T04:32:41.681 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00146148 2024-04-26T04:32:41.681 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.681 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.681 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.682 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.513012+0000", 2024-04-26T04:32:41.682 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.1072000000000001e-05 2024-04-26T04:32:41.682 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.682 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.682 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.682 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.683 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.683 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 35/30 ECSubWrite(tid=34, reqid=client.4670.0:202, at_version=35'24, trim_to=0'0, roll_forward_to=35'22))", 2024-04-26T04:32:41.683 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:29.161802+0000", 2024-04-26T04:32:41.683 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 12.451594434, 2024-04-26T04:32:41.683 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103248995, 2024-04-26T04:32:41.683 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.684 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.684 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.684 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.684 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.684 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.684 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.161802+0000", 2024-04-26T04:32:41.685 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.685 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.685 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.685 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.685 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.161802+0000", 2024-04-26T04:32:41.685 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.686 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.686 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.686 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.686 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.161804+0000", 2024-04-26T04:32:41.686 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9539999999999998e-06 2024-04-26T04:32:41.686 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.687 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.687 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.687 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.162608+0000", 2024-04-26T04:32:41.687 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00080376299999999998 2024-04-26T04:32:41.687 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.687 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.688 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.688 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.162641+0000", 2024-04-26T04:32:41.688 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2570000000000002e-05 2024-04-26T04:32:41.688 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.688 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.688 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.688 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.162665+0000", 2024-04-26T04:32:41.689 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4939999999999998e-05 2024-04-26T04:32:41.689 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.689 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.689 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.689 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.262947+0000", 2024-04-26T04:32:41.689 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10028109 2024-04-26T04:32:41.690 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.690 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.690 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.690 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.262963+0000", 2024-04-26T04:32:41.690 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6152999999999999e-05 2024-04-26T04:32:41.690 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.690 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.690 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.690 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.264973+0000", 2024-04-26T04:32:41.690 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020107390000000001 2024-04-26T04:32:41.690 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.691 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.691 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.691 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.265051+0000", 2024-04-26T04:32:41.691 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7786000000000004e-05 2024-04-26T04:32:41.691 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.691 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.691 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.691 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:280 3.0s0 3:0061323f:::smithi17829308-8:head [delete] snapc 2=[2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.865956+0000", 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.747440534000001, 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20424141400000001, 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 280 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.692 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865956+0000", 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865956+0000", 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865959+0000", 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0570000000000001e-06 2024-04-26T04:32:41.693 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.694 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.694 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.694 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865971+0000", 2024-04-26T04:32:41.694 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1576000000000001e-05 2024-04-26T04:32:41.694 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.694 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.694 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.694 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866003+0000", 2024-04-26T04:32:41.694 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2543999999999999e-05 2024-04-26T04:32:41.694 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.694 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.694 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.694 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866020+0000", 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6433e-05 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866157+0000", 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013767200000000001 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866327+0000", 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016995199999999999 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.695 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.867476+0000", 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0011486700000000001 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.869167+0000", 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016912450000000001 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.070080+0000", 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200912916 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.696 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.070198+0000", 2024-04-26T04:32:41.697 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000117349 2024-04-26T04:32:41.697 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.697 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.697 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.697 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.697 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.697 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs2 36/30 ECSubWrite(tid=35, reqid=client.4670.0:281, at_version=36'18, trim_to=0'0, roll_forward_to=35'16))", 2024-04-26T04:32:41.697 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:31.077869+0000", 2024-04-26T04:32:41.697 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.535528151999999, 2024-04-26T04:32:41.697 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10269748500000001, 2024-04-26T04:32:41.697 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.697 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.698 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.698 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.698 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.698 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.698 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.077869+0000", 2024-04-26T04:32:41.698 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.698 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.698 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.698 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.698 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.077869+0000", 2024-04-26T04:32:41.698 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.698 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.699 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.699 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.699 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.077881+0000", 2024-04-26T04:32:41.699 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2374e-05 2024-04-26T04:32:41.699 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.699 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.699 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.699 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.078021+0000", 2024-04-26T04:32:41.699 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00014036800000000001 2024-04-26T04:32:41.699 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.699 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.699 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.700 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.078050+0000", 2024-04-26T04:32:41.700 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9087000000000001e-05 2024-04-26T04:32:41.700 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.700 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.700 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.700 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.178190+0000", 2024-04-26T04:32:41.701 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100139807 2024-04-26T04:32:41.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.701 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.701 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.701 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.178379+0000", 2024-04-26T04:32:41.701 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000188522 2024-04-26T04:32:41.701 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.701 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.701 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.701 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.178394+0000", 2024-04-26T04:32:41.701 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5362999999999998e-05 2024-04-26T04:32:41.702 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.702 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.702 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.702 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.180507+0000", 2024-04-26T04:32:41.702 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021130179999999999 2024-04-26T04:32:41.702 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.702 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.702 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.702 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.180566+0000", 2024-04-26T04:32:41.702 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8946000000000001e-05 2024-04-26T04:32:41.702 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.702 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.702 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.as4 37/30 ECSubWrite(tid=19, reqid=client.4670.0:285, at_version=37'12, trim_to=0'0, roll_forward_to=36'10))", 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:31.258576+0000", 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.354820198000001, 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103210529, 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.258576+0000", 2024-04-26T04:32:41.703 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.704 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.704 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.704 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.704 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.258576+0000", 2024-04-26T04:32:41.704 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.704 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.704 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.704 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.704 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.258577+0000", 2024-04-26T04:32:41.705 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.53e-07 2024-04-26T04:32:41.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.705 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.705 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.705 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.259163+0000", 2024-04-26T04:32:41.705 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00058609799999999996 2024-04-26T04:32:41.705 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.705 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.705 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.705 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.259193+0000", 2024-04-26T04:32:41.706 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9623000000000001e-05 2024-04-26T04:32:41.706 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.706 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.706 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.706 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.259209+0000", 2024-04-26T04:32:41.706 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6303999999999998e-05 2024-04-26T04:32:41.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.707 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.707 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.707 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.359488+0000", 2024-04-26T04:32:41.707 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100279481 2024-04-26T04:32:41.707 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.708 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.708 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.708 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.359503+0000", 2024-04-26T04:32:41.708 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4514e-05 2024-04-26T04:32:41.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.708 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.708 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.708 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.361727+0000", 2024-04-26T04:32:41.708 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0022245870000000001 2024-04-26T04:32:41.708 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.708 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.708 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.361787+0000", 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9568999999999998e-05 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds2 40/30 ECSubWrite(tid=32, reqid=client.4670.0:332, at_version=40'19, trim_to=0'0, roll_forward_to=40'17))", 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.920443+0000", 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.6929533970000001, 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102952369, 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.709 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920443+0000", 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920443+0000", 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920445+0000", 2024-04-26T04:32:41.710 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.967e-06 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920497+0000", 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1675999999999998e-05 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920535+0000", 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7846000000000003e-05 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.711 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.020672+0000", 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100137371 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.020893+0000", 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00022085099999999999 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.020917+0000", 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3703e-05 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.712 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023292+0000", 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002375463 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023396+0000", 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000103492 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds2 40/30 ECSubWrite(tid=33, reqid=client.4670.0:332, at_version=0'0, trim_to=0'0, roll_forward_to=40'19))", 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:35.023904+0000", 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.589492312, 2024-04-26T04:32:41.713 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201717813, 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023904+0000", 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023904+0000", 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.714 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.715 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.715 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.715 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023906+0000", 2024-04-26T04:32:41.715 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.387e-06 2024-04-26T04:32:41.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.715 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.715 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.715 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023925+0000", 2024-04-26T04:32:41.715 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9059000000000002e-05 2024-04-26T04:32:41.715 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.715 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.715 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.715 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023952+0000", 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7455999999999999e-05 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023968+0000", 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.526e-05 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.224267+0000", 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200299636 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.716 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.224282+0000", 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5007e-05 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.225547+0000", 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012650719999999999 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.225622+0000", 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4936000000000002e-05 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.717 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.4s5 45/30 ECSubWrite(tid=34, reqid=client.4670.0:377, at_version=45'23, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:39.634866+0000", 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 1.978530683, 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20147746799999999, 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634866+0000", 2024-04-26T04:32:41.718 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634866+0000", 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634866+0000", 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8599999999999999e-07 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:41.719 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634876+0000", 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8649999999999996e-06 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634884+0000", 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7060000000000004e-06 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.734989+0000", 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100104926 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:41.720 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.835213+0000", 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100223672 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.835223+0000", 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0570999999999999e-05 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.836270+0000", 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0010470220000000001 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:41.721 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:41.722 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.836343+0000", 2024-04-26T04:32:41.722 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3319999999999996e-05 2024-04-26T04:32:41.722 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.722 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.722 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.722 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:41.722 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:41.722 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:41.724 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T04:32:41.724 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:41.722+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:41.785 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:41.786 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:41.812 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:41.812 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:41.825 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:41.821+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:41.838 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:41.838 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:41.854 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:41.854 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:41.855 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 0 2024-04-26T04:32:41.855 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:41.865 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:41.866 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:41.869 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T04:32:41.892 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:41.893 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:41.919 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:41.920 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:41.925 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:41.922+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:41.947 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:41.948 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:41.974 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:41.974 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:41.990 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:41.990 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:41.998 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:41.999 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:42.004 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T04:32:42.026 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:42.026 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:42.027 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:42.025+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:42.053 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:42.053 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:42.078 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:42.078 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:42.101 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:42.101 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:42.127 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:42.128 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:42.128 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:42.124+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:42.152 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:42.152 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:42.165 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:42.166 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:42.166 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.4 to 1 (10000) 2024-04-26T04:32:42.174 INFO:tasks.rados.rados.0.smithi178.stdout:147: done (1 left) 2024-04-26T04:32:42.174 INFO:tasks.rados.rados.0.smithi178.stdout:153: done (0 left) 2024-04-26T04:32:42.174 INFO:tasks.rados.rados.0.smithi178.stdout:154: read oid 46 snap -1 2024-04-26T04:32:42.175 INFO:tasks.rados.rados.0.smithi178.stdout:154: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-26T04:32:42.175 INFO:tasks.rados.rados.0.smithi178.stdout:155: copy_from oid 3 from oid 30 current snap is 8 2024-04-26T04:32:42.175 INFO:tasks.rados.rados.0.smithi178.stdout:156: copy_from oid 17 from oid 9 current snap is 8 2024-04-26T04:32:42.175 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 31 current snap is 8 2024-04-26T04:32:42.175 INFO:tasks.rados.rados.0.smithi178.stdout:157: seq_num 78 ranges {2949120=557056,3506176=671744,4177920=98304} 2024-04-26T04:32:42.180 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:42.181 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:42.181 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T04:32:42.195 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-26T04:32:42.205 INFO:tasks.rados.rados.0.smithi178.stdout:157: writing smithi17829308-31 from 2949120 to 3506176 tid 1 2024-04-26T04:32:42.210 INFO:tasks.rados.rados.0.smithi178.stdout:157: writing smithi17829308-31 from 3506176 to 4177920 tid 2 2024-04-26T04:32:42.211 INFO:tasks.rados.rados.0.smithi178.stdout:157: writing smithi17829308-31 from 4177920 to 4276224 tid 3 2024-04-26T04:32:42.211 INFO:tasks.rados.rados.0.smithi178.stdout:158: setattr oid 32 current snap is 8 2024-04-26T04:32:42.213 INFO:tasks.rados.rados.0.smithi178.stdout:159: snap_remove snap 3 2024-04-26T04:32:42.248 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:42.245+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:42.299 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:42.299 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:42.314 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T04:32:42.329 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:42.325+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:42.430 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:42.428+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:42.481 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:42.481 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:42.495 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T04:32:42.531 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:42.528+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:42.613 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:42.613 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:42.627 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T04:32:42.671 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:42.670+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:42.733 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:42.728+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:42.750 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:42.750 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:42.764 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T04:32:42.834 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:42.833+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:42.930 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:42.931 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:42.935 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:42.932+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:42.944 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T04:32:43.036 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:43.034+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:43.068 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:43.068 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:43.081 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T04:32:43.138 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:43.136+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:43.169 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.5 to 1 (10000) 2024-04-26T04:32:43.175 INFO:tasks.rados.rados.0.smithi178.stdout:159: done (5 left) 2024-04-26T04:32:43.175 INFO:tasks.rados.rados.0.smithi178.stdout:160: rmattr oid 38 current snap is 8 2024-04-26T04:32:43.175 INFO:tasks.rados.rados.0.smithi178.stdout:160: done (5 left) 2024-04-26T04:32:43.175 INFO:tasks.rados.rados.0.smithi178.stdout:161: copy_from oid 43 from oid 35 current snap is 8 2024-04-26T04:32:43.175 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 32 v 45 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-04-26T04:32:43.175 INFO:tasks.rados.rados.0.smithi178.stdout:156: finishing copy_from racing read to smithi17829308-17 2024-04-26T04:32:43.175 INFO:tasks.rados.rados.0.smithi178.stdout:157: finishing write tid 1 to smithi17829308-31 2024-04-26T04:32:43.175 INFO:tasks.rados.rados.0.smithi178.stdout:157: finishing write tid 2 to smithi17829308-31 2024-04-26T04:32:43.176 INFO:tasks.rados.rados.0.smithi178.stdout:156: finishing copy_from to smithi17829308-17 2024-04-26T04:32:43.176 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 17 v 21 (ObjNum 63 snap 2 seq_num 63) dirty exists 2024-04-26T04:32:43.176 INFO:tasks.rados.rados.0.smithi178.stdout:157: finishing write tid 3 to smithi17829308-31 2024-04-26T04:32:43.176 INFO:tasks.rados.rados.0.smithi178.stdout:154: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-26T04:32:43.189 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-26T04:32:43.197 INFO:tasks.rados.rados.0.smithi178.stdout:157: finishing write tid 4 to smithi17829308-31 2024-04-26T04:32:43.197 INFO:tasks.rados.rados.0.smithi178.stdout:157: finishing write tid 5 to smithi17829308-31 2024-04-26T04:32:43.197 INFO:tasks.rados.rados.0.smithi178.stdout:157: oid 31 updating version 0 to 25 2024-04-26T04:32:43.198 INFO:tasks.rados.rados.0.smithi178.stdout:157: oid 31 updating version 25 to 27 2024-04-26T04:32:43.198 INFO:tasks.rados.rados.0.smithi178.stdout:157: oid 31 updating version 27 to 28 2024-04-26T04:32:43.198 INFO:tasks.rados.rados.0.smithi178.stdout:157: oid 31 version 28 is already newer than 26 2024-04-26T04:32:43.198 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 31 v 28 (ObjNum 78 snap 8 seq_num 78) dirty exists 2024-04-26T04:32:43.198 INFO:tasks.rados.rados.0.smithi178.stdout:157: left oid 31 (ObjNum 78 snap 8 seq_num 78) 2024-04-26T04:32:43.198 INFO:tasks.rados.rados.0.smithi178.stdout:155: finishing copy_from racing read to smithi17829308-3 2024-04-26T04:32:43.198 INFO:tasks.rados.rados.0.smithi178.stdout:154: done (5 left) 2024-04-26T04:32:43.198 INFO:tasks.rados.rados.0.smithi178.stdout:156: done (4 left) 2024-04-26T04:32:43.198 INFO:tasks.rados.rados.0.smithi178.stdout:157: done (3 left) 2024-04-26T04:32:43.198 INFO:tasks.rados.rados.0.smithi178.stdout:158: done (2 left) 2024-04-26T04:32:43.198 INFO:tasks.rados.rados.0.smithi178.stdout:162: snap_remove snap 4 2024-04-26T04:32:43.199 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-26T04:32:43.202 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:43.202 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:43.217 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T04:32:43.263 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:43.262+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:43.337 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:43.337 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:43.339 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:43.337+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:43.351 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T04:32:43.443 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:43.442+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:43.523 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:43.524 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:43.538 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T04:32:43.579 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:43.578+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:43.642 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:43.641+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:43.664 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:43.664 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:43.682 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T04:32:43.742 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:43.737+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:43.844 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:43.840+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:43.849 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:43.850 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:43.864 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T04:32:43.944 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:43.937+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:43.990 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:43.990 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:44.005 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T04:32:44.053 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:44.053+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:44.131 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:44.131 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:44.146 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T04:32:44.146 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:44.145+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-04-26T04:32:44.176 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.6 to 1 (10000) 2024-04-26T04:32:44.186 INFO:tasks.rados.rados.0.smithi178.stdout:162: done (2 left) 2024-04-26T04:32:44.186 INFO:tasks.rados.rados.0.smithi178.stdout:163: read oid 4 snap -1 2024-04-26T04:32:44.186 INFO:tasks.rados.rados.0.smithi178.stdout:163: expect (ObjNum 3 snap 0 seq_num 3) 2024-04-26T04:32:44.186 INFO:tasks.rados.rados.0.smithi178.stdout:164: rollback oid 25 current snap is 8 2024-04-26T04:32:44.186 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 25 to 7 2024-04-26T04:32:44.186 INFO:tasks.rados.rados.0.smithi178.stdout:155: finishing copy_from to smithi17829308-3 2024-04-26T04:32:44.186 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 3 v 2 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-26T04:32:44.186 INFO:tasks.rados.rados.0.smithi178.stdout:155: done (3 left) 2024-04-26T04:32:44.186 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 45 current snap is 8 2024-04-26T04:32:44.186 INFO:tasks.rados.rados.0.smithi178.stdout:161: finishing copy_from racing read to smithi17829308-43 2024-04-26T04:32:44.187 INFO:tasks.rados.rados.0.smithi178.stdout:165: seq_num 79 ranges {0=573440,573440=425984,999424=344064} 2024-04-26T04:32:44.194 INFO:tasks.rados.rados.0.smithi178.stdout:165: writing smithi17829308-45 from 0 to 573440 tid 1 2024-04-26T04:32:44.199 INFO:tasks.rados.rados.0.smithi178.stdout:165: writing smithi17829308-45 from 573440 to 999424 tid 2 2024-04-26T04:32:44.203 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-26T04:32:44.203 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T04:32:44.204 INFO:tasks.rados.rados.0.smithi178.stdout:165: writing smithi17829308-45 from 999424 to 1343488 tid 3 2024-04-26T04:32:44.205 INFO:tasks.rados.rados.0.smithi178.stdout:166: read oid 42 snap -1 2024-04-26T04:32:44.205 INFO:tasks.rados.rados.0.smithi178.stdout:166: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:32:44.205 INFO:tasks.rados.rados.0.smithi178.stdout:167: rollback oid 48 current snap is 8 2024-04-26T04:32:44.205 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 48 to 7 2024-04-26T04:32:44.205 INFO:tasks.rados.rados.0.smithi178.stdout:161: finishing copy_from to smithi17829308-43 2024-04-26T04:32:44.205 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 43 v 29 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-04-26T04:32:44.205 INFO:tasks.rados.rados.0.smithi178.stdout:164: finishing rollback tid 1 to smithi17829308-25 2024-04-26T04:32:44.205 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 25 v 0 (ObjNum 129 snap 0 seq_num 2234020936) dirty dne 2024-04-26T04:32:44.205 INFO:tasks.rados.rados.0.smithi178.stdout:161: done (5 left) 2024-04-26T04:32:44.205 INFO:tasks.rados.rados.0.smithi178.stdout:164: done (4 left) 2024-04-26T04:32:44.205 INFO:tasks.rados.rados.0.smithi178.stdout:168: delete oid 1 current snap is 8 2024-04-26T04:32:44.246 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:44.245+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:44.304 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:44.304 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:44.324 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T04:32:44.366 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:44.365+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:44.390 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:44.391 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:44.414 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:44.414 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:44.417 INFO:tasks.rados.rados.0.smithi178.stdout:167: finishing rollback tid 0 to smithi17829308-48 2024-04-26T04:32:44.435 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:44.435 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:44.448 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:44.444+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:44.456 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:44.456 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:44.486 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:44.486 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:44.503 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T04:32:44.512 INFO:tasks.rados.rados.0.smithi178.stdout:166: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:32:44.550 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:44.549+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:44.561 INFO:tasks.rados.rados.0.smithi178.stdout:167: finishing rollback tid 1 to smithi17829308-48 2024-04-26T04:32:44.592 INFO:tasks.rados.rados.0.smithi178.stdout:163: expect (ObjNum 3 snap 0 seq_num 3) 2024-04-26T04:32:44.613 INFO:tasks.rados.rados.0.smithi178.stdout:167: finishing rollback tid 2 to smithi17829308-48 2024-04-26T04:32:44.613 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 48 v 32 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-04-26T04:32:44.621 INFO:tasks.rados.rados.0.smithi178.stdout:165: finishing write tid 1 to smithi17829308-45 2024-04-26T04:32:44.630 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:44.630 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:44.650 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:44.649+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:44.652 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:44.652 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:44.663 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:44.663 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:44.675 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:44.675 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:44.676 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T04:32:44.702 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:44.703 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:44.722 INFO:tasks.rados.rados.0.smithi178.stdout:165: finishing write tid 2 to smithi17829308-45 2024-04-26T04:32:44.722 INFO:tasks.rados.rados.0.smithi178.stdout:165: finishing write tid 3 to smithi17829308-45 2024-04-26T04:32:44.722 INFO:tasks.rados.rados.0.smithi178.stdout:165: finishing write tid 4 to smithi17829308-45 2024-04-26T04:32:44.727 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:44.728 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:44.751 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:44.752+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:44.752 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:44.752 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:44.774 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:44.774 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:44.796 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:44.796 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:44.816 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:44.817 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:44.828 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:44.829 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:32:44.829 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-04-26T04:32:44.829 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-04-26T04:32:44.829 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:44.839 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:44.839 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:44.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T04:32:44.852 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:44.848+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:44.862 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:44.863 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:44.887 INFO:tasks.rados.rados.0.smithi178.stdout:163: done (4 left) 2024-04-26T04:32:44.887 INFO:tasks.rados.rados.0.smithi178.stdout:166: done (3 left) 2024-04-26T04:32:44.887 INFO:tasks.rados.rados.0.smithi178.stdout:167: done (2 left) 2024-04-26T04:32:44.887 INFO:tasks.rados.rados.0.smithi178.stdout:168: done (1 left) 2024-04-26T04:32:44.888 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 48 current snap is 8 2024-04-26T04:32:44.888 INFO:tasks.rados.rados.0.smithi178.stdout:169: seq_num 81 ranges {802816=606208,1409024=720896,2129920=49152} 2024-04-26T04:32:44.910 INFO:tasks.rados.rados.0.smithi178.stdout:169: writing smithi17829308-48 from 802816 to 1409024 tid 1 2024-04-26T04:32:44.920 INFO:tasks.rados.rados.0.smithi178.stdout:169: writing smithi17829308-48 from 1409024 to 2129920 tid 2 2024-04-26T04:32:44.921 INFO:tasks.rados.rados.0.smithi178.stdout:169: writing smithi17829308-48 from 2129920 to 2179072 tid 3 2024-04-26T04:32:44.921 INFO:tasks.rados.rados.0.smithi178.stdout:170: snap_create 2024-04-26T04:32:44.925 INFO:tasks.rados.rados.0.smithi178.stdout:165: finishing write tid 5 to smithi17829308-45 2024-04-26T04:32:44.925 INFO:tasks.rados.rados.0.smithi178.stdout:165: oid 45 updating version 0 to 36 2024-04-26T04:32:44.925 INFO:tasks.rados.rados.0.smithi178.stdout:165: oid 45 updating version 36 to 37 2024-04-26T04:32:44.925 INFO:tasks.rados.rados.0.smithi178.stdout:165: oid 45 updating version 37 to 39 2024-04-26T04:32:44.925 INFO:tasks.rados.rados.0.smithi178.stdout:165: oid 45 version 39 is already newer than 38 2024-04-26T04:32:44.925 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 45 v 39 (ObjNum 79 snap 8 seq_num 79) dirty exists 2024-04-26T04:32:44.925 INFO:tasks.rados.rados.0.smithi178.stdout:165: left oid 45 (ObjNum 79 snap 8 seq_num 79) 2024-04-26T04:32:44.949 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:44.949 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:44.953 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:44.949+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:44.985 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:44.985 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-04-26T04:32:44.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-04-26T04:32:44.985 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:32:44.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.985 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:10 3.4s0 3:2b198661:::smithi17829308-4:head [append 0~65536,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:44.985 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.199689+0000", 2024-04-26T04:32:44.985 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.781367638999999, 2024-04-26T04:32:44.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.209779622, 2024-04-26T04:32:44.986 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:44.986 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:44.986 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:44.986 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:44.986 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:44.986 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:44.986 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 10 2024-04-26T04:32:44.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.986 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:44.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:44.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.199689+0000", 2024-04-26T04:32:44.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:44.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:44.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.199689+0000", 2024-04-26T04:32:44.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:44.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:44.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.199691+0000", 2024-04-26T04:32:44.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2330000000000001e-06 2024-04-26T04:32:44.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:44.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.199930+0000", 2024-04-26T04:32:44.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023908999999999999 2024-04-26T04:32:44.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:44.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.199970+0000", 2024-04-26T04:32:44.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9366999999999998e-05 2024-04-26T04:32:44.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:44.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.199993+0000", 2024-04-26T04:32:44.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2874000000000002e-05 2024-04-26T04:32:44.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:44.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200190+0000", 2024-04-26T04:32:44.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019786900000000001 2024-04-26T04:32:44.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:44.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200577+0000", 2024-04-26T04:32:44.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00038680000000000002 2024-04-26T04:32:44.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:44.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.202134+0000", 2024-04-26T04:32:44.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0015565519999999999 2024-04-26T04:32:44.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:44.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.208673+0000", 2024-04-26T04:32:44.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0065391490000000002 2024-04-26T04:32:44.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:32:44.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.409345+0000", 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20067243400000001 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.409469+0000", 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000123254 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:11 3.4s0 3:2b198661:::smithi17829308-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.200047+0000", 2024-04-26T04:32:44.991 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.781010071000001, 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.30557167699999999, 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 11 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200047+0000", 2024-04-26T04:32:44.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200047+0000", 2024-04-26T04:32:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200048+0000", 2024-04-26T04:32:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.747e-06 2024-04-26T04:32:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:44.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200059+0000", 2024-04-26T04:32:44.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.06e-05 2024-04-26T04:32:44.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:44.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200083+0000", 2024-04-26T04:32:44.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4307e-05 2024-04-26T04:32:44.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:44.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200101+0000", 2024-04-26T04:32:44.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8050000000000002e-05 2024-04-26T04:32:44.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.204381+0000", 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0042801339999999997 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.204634+0000", 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025263999999999999 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.205920+0000", 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0012855220000000001 2024-04-26T04:32:44.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:44.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.309119+0000", 2024-04-26T04:32:44.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.103199077 2024-04-26T04:32:44.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:32:44.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.505436+0000", 2024-04-26T04:32:44.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.196317617 2024-04-26T04:32:44.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:44.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.505618+0000", 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018198300000000001 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpRead(3.4s0 33/30 ECSubRead(tid=6, to_read={3:2b198661:::smithi17829308-4:head=0,4096,0}, subchunks={3:2b198661:::smithi17829308-4:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.506484+0000", 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 19.474573065000001, 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.21111134600000001, 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:44.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.506484+0000", 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.506484+0000", 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.506484+0000", 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:44.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.506484+0000", 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.506587+0000", 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000103022 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.606688+0000", 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100101886 2024-04-26T04:32:44.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.717209+0000", 2024-04-26T04:32:45.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.11052094899999999 2024-04-26T04:32:45.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.717595+0000", 2024-04-26T04:32:45.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00038548899999999998 2024-04-26T04:32:45.000 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.000 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.000 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.001 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(osd.4.21:5 3.4s0 3:26eb43ca:::smithi17829308-16:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:45.001 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.182301+0000", 2024-04-26T04:32:45.001 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.7987556819999995, 2024-04-26T04:32:45.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.391615035, 2024-04-26T04:32:45.001 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.001 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.001 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:45.001 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.182301+0000", 2024-04-26T04:32:45.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.182301+0000", 2024-04-26T04:32:45.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.182304+0000", 2024-04-26T04:32:45.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8930000000000001e-06 2024-04-26T04:32:45.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.182316+0000", 2024-04-26T04:32:45.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.189e-05 2024-04-26T04:32:45.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.182343+0000", 2024-04-26T04:32:45.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7174000000000001e-05 2024-04-26T04:32:45.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.182365+0000", 2024-04-26T04:32:45.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2240000000000001e-05 2024-04-26T04:32:45.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.182504+0000", 2024-04-26T04:32:45.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013861799999999999 2024-04-26T04:32:45.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:45.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.182656+0000", 2024-04-26T04:32:45.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015260100000000001 2024-04-26T04:32:45.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.573916+0000", 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.391259619 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:374 3.4s0 3:26eb43ca:::smithi17829308-16:head [append 0~507904] snapc 8=[8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.421498+0000", 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.5595581159999998, 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50938256599999998, 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.005 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:45.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:45.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:45.006 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:45.006 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 374 2024-04-26T04:32:45.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.006 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.421498+0000", 2024-04-26T04:32:45.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.421498+0000", 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.421501+0000", 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3199999999999998e-06 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.422203+0000", 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00070210900000000002 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.422223+0000", 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9745000000000002e-05 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.422232+0000", 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3260000000000002e-06 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.422300+0000", 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7710000000000001e-05 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op must wait for map", 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.422305+0000", 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1540000000000001e-06 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.528615+0000", 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.106310432 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.528743+0000", 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000127637 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530006+0000", 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0012630670000000001 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.533919+0000", 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0039127679999999996 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.930754+0000", 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39683563700000002 2024-04-26T04:32:45.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.930881+0000", 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000126661 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:375 3.4s0 3:26eb43ca:::smithi17829308-16:head [append 0~507904] snapc 8=[8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.427459+0000", 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.5535975129999997, 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.60387316999999996, 2024-04-26T04:32:45.011 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.012 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.012 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:45.012 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:45.012 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:45.012 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:45.012 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 375 2024-04-26T04:32:45.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.012 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.427459+0000", 2024-04-26T04:32:45.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.427459+0000", 2024-04-26T04:32:45.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.427460+0000", 2024-04-26T04:32:45.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0499999999999999e-06 2024-04-26T04:32:45.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.428039+0000", 2024-04-26T04:32:45.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057915600000000005 2024-04-26T04:32:45.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.428054+0000", 2024-04-26T04:32:45.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4941e-05 2024-04-26T04:32:45.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.428061+0000", 2024-04-26T04:32:45.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8129999999999998e-06 2024-04-26T04:32:45.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430497+0000", 2024-04-26T04:32:45.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002436405 2024-04-26T04:32:45.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:45.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430504+0000", 2024-04-26T04:32:45.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0299999999999996e-06 2024-04-26T04:32:45.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.015 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530949+0000", 2024-04-26T04:32:45.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100444931 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.531063+0000", 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0001138 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.531997+0000", 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00093402000000000003 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.533975+0000", 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0019779659999999998 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.031211+0000", 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49723574799999998 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.031332+0000", 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012131 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.017 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.018 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:376 3.4s0 3:26eb43ca:::smithi17829308-16:head [append 0~262144] snapc 8=[8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:32:45.018 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.430172+0000", 2024-04-26T04:32:45.018 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.5508849419999997, 2024-04-26T04:32:45.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.70162850899999996, 2024-04-26T04:32:45.018 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.018 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.018 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:45.018 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:45.018 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:45.018 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 376 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430172+0000", 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430172+0000", 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430173+0000", 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1200000000000001e-07 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430402+0000", 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022931899999999999 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430418+0000", 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6401000000000001e-05 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430431+0000", 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2278e-05 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430523+0000", 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2398999999999994e-05 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430526+0000", 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9809999999999999e-06 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.532597+0000", 2024-04-26T04:32:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.102070991 2024-04-26T04:32:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.532697+0000", 2024-04-26T04:32:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100453 2024-04-26T04:32:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.533465+0000", 2024-04-26T04:32:45.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00076714800000000005 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.534877+0000", 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00141208 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.131673+0000", 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.59679601000000004 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.131800+0000", 2024-04-26T04:32:45.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000127537 2024-04-26T04:32:45.024 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.024 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.024 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.024 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:377 3.4s0 3:26eb43ca:::smithi17829308-16:head [setxattr _header (56) in=7b] snapc 8=[8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:32:45.024 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.430444+0000", 2024-04-26T04:32:45.024 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.5506121520000002, 2024-04-26T04:32:45.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.70171558099999998, 2024-04-26T04:32:45.024 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.024 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.024 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 377 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430444+0000", 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430444+0000", 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430445+0000", 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.68e-07 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430450+0000", 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.493e-06 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430463+0000", 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3378e-05 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430470+0000", 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1740000000000003e-06 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.504874+0000", 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.074404153000000001 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.504878+0000", 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9759999999999997e-06 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.534051+0000", 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029172849000000001 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.534150+0000", 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.857e-05 2024-04-26T04:32:45.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.534617+0000", 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046738200000000002 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.535807+0000", 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011894379999999999 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.132031+0000", 2024-04-26T04:32:45.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.59622439299999996 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.132160+0000", 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000129107 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:378 3.4s0 3:26eb43ca:::smithi17829308-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.430481+0000", 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.5505752390000005, 2024-04-26T04:32:45.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1057942590000001, 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 378 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430481+0000", 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430481+0000", 2024-04-26T04:32:45.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430482+0000", 2024-04-26T04:32:45.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4000000000000002e-07 2024-04-26T04:32:45.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430484+0000", 2024-04-26T04:32:45.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.52e-06 2024-04-26T04:32:45.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430493+0000", 2024-04-26T04:32:45.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9139999999999997e-06 2024-04-26T04:32:45.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530567+0000", 2024-04-26T04:32:45.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10007361300000001 2024-04-26T04:32:45.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.535742+0000", 2024-04-26T04:32:45.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0051753149999999998 2024-04-26T04:32:45.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:45.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.535800+0000", 2024-04-26T04:32:45.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7997000000000001e-05 2024-04-26T04:32:45.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.034 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.138278+0000", 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.60247765099999995 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.138452+0000", 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000173618 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.536276+0000", 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39782409099999999 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:379 3.4s0 3:25070d22:::smithi17829308-10:head [copy-from ver 2] snapc 8=[8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.530582+0000", 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.4504742300000002, 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1058113540000001, 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 379 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530582+0000", 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530582+0000", 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530584+0000", 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.314e-06 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530589+0000", 2024-04-26T04:32:45.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6189999999999999e-06 2024-04-26T04:32:45.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530598+0000", 2024-04-26T04:32:45.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2979999999999992e-06 2024-04-26T04:32:45.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.630723+0000", 2024-04-26T04:32:45.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10012523199999999 2024-04-26T04:32:45.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.930085+0000", 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.29936250800000003 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.930252+0000", 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000166665 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.136077+0000", 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20582483400000001 2024-04-26T04:32:45.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.139888+0000", 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00381131 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.636327+0000", 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49643924299999997 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.636394+0000", 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6330999999999999e-05 2024-04-26T04:32:45.040 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 45/30 ECSubWriteReply(tid=33, last_complete=45'22, committed=1, applied=1))", 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.556947+0000", 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.424109906, 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.27243553100000001, 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.556947+0000", 2024-04-26T04:32:45.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.556947+0000", 2024-04-26T04:32:45.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.556947+0000", 2024-04-26T04:32:45.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9500000000000003e-07 2024-04-26T04:32:45.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.556950+0000", 2024-04-26T04:32:45.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2959999999999999e-06 2024-04-26T04:32:45.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.556959+0000", 2024-04-26T04:32:45.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.208e-06 2024-04-26T04:32:45.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.556963+0000", 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4569999999999998e-06 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.829352+0000", 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.27238926800000002 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.829382+0000", 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9807000000000001e-05 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.044 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 45/30 ECSubWriteReply(tid=34, last_complete=45'23, committed=1, applied=1))", 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.557248+0000", 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.4238081510000002, 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.27232231400000001, 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.557248+0000", 2024-04-26T04:32:45.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.557248+0000", 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.557249+0000", 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.46e-07 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.557252+0000", 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0570000000000001e-06 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.557260+0000", 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.8860000000000004e-06 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.557264+0000", 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.143e-06 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.829533+0000", 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.27226948499999998 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.829571+0000", 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7296999999999999e-05 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:380 3.4s0 3:25070d22:::smithi17829308-10:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.630758+0000", 2024-04-26T04:32:45.048 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.3502986669999997, 2024-04-26T04:32:45.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50899852300000004, 2024-04-26T04:32:45.049 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.049 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.049 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:45.049 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:45.049 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:45.049 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:45.049 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 380 2024-04-26T04:32:45.049 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.049 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.049 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.630758+0000", 2024-04-26T04:32:45.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.630758+0000", 2024-04-26T04:32:45.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.630760+0000", 2024-04-26T04:32:45.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.956e-06 2024-04-26T04:32:45.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.630770+0000", 2024-04-26T04:32:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0471e-05 2024-04-26T04:32:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.630792+0000", 2024-04-26T04:32:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.143e-05 2024-04-26T04:32:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.630803+0000", 2024-04-26T04:32:45.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.094e-05 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.139422+0000", 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50861925399999997 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.139602+0000", 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018024499999999999 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.139756+0000", 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000154227 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 45/30 ECSubWriteReply(tid=33, last_complete=45'22, committed=1, applied=1))", 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.835454+0000", 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.145602856, 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.296415753, 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.053 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.835454+0000", 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.835454+0000", 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.835455+0000", 2024-04-26T04:32:45.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3230000000000001e-06 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.835464+0000", 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3470000000000005e-06 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.835496+0000", 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1157e-05 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.835514+0000", 2024-04-26T04:32:45.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8263000000000001e-05 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.131547+0000", 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.29603310500000002 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.131869+0000", 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032255800000000002 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 45/30 ECSubWriteReply(tid=34, last_complete=45'23, committed=1, applied=1))", 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.836727+0000", 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.1443297350000003, 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.29618684099999998, 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.836727+0000", 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.836727+0000", 2024-04-26T04:32:45.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.058 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.836728+0000", 2024-04-26T04:32:45.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0139999999999999e-06 2024-04-26T04:32:45.058 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.058 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.058 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.836735+0000", 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1300000000000003e-06 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.836756+0000", 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1426000000000001e-05 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.836769+0000", 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2597e-05 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.131922+0000", 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.29515309200000001 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.132914+0000", 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00099158199999999997 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 46/30 ECSubWriteReply(tid=39, last_complete=46'27, committed=1, applied=1))", 2024-04-26T04:32:45.060 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:40.841612+0000", 2024-04-26T04:32:45.061 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.1394449509999998, 2024-04-26T04:32:45.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.30065419500000001, 2024-04-26T04:32:45.061 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.061 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.061 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.061 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.841612+0000", 2024-04-26T04:32:45.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.841612+0000", 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.841614+0000", 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.956e-06 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.841629+0000", 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5780000000000001e-05 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.841667+0000", 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.769e-05 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.941760+0000", 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100092536 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.142202+0000", 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20044268600000001 2024-04-26T04:32:45.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.142266+0000", 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3547000000000007e-05 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 46/30 ECSubWriteReply(tid=40, last_complete=46'29, committed=1, applied=1))", 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:40.943865+0000", 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.0371915879999998, 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.30067569199999999, 2024-04-26T04:32:45.064 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.943865+0000", 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.943865+0000", 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.943867+0000", 2024-04-26T04:32:45.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.717e-06 2024-04-26T04:32:45.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.943878+0000", 2024-04-26T04:32:45.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1153e-05 2024-04-26T04:32:45.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.943924+0000", 2024-04-26T04:32:45.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6057000000000003e-05 2024-04-26T04:32:45.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.943939+0000", 2024-04-26T04:32:45.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5475999999999999e-05 2024-04-26T04:32:45.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.243817+0000", 2024-04-26T04:32:45.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.299877165 2024-04-26T04:32:45.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.244541+0000", 2024-04-26T04:32:45.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00072412400000000003 2024-04-26T04:32:45.068 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.068 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.068 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.068 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.068 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 46/30 ECSubWriteReply(tid=40, last_complete=46'29, committed=1, applied=1))", 2024-04-26T04:32:45.068 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:40.944025+0000", 2024-04-26T04:32:45.068 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.0370319339999998, 2024-04-26T04:32:45.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.29972996800000001, 2024-04-26T04:32:45.068 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.944025+0000", 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.944025+0000", 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.944026+0000", 2024-04-26T04:32:45.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1299999999999998e-07 2024-04-26T04:32:45.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.944031+0000", 2024-04-26T04:32:45.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6540000000000004e-06 2024-04-26T04:32:45.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.944050+0000", 2024-04-26T04:32:45.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8527000000000001e-05 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.944060+0000", 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0283e-05 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.243717+0000", 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.29965658000000001 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.071 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.243755+0000", 2024-04-26T04:32:45.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8010999999999999e-05 2024-04-26T04:32:45.072 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.072 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.072 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.072 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 46/30 ECSubWriteReply(tid=40, last_complete=46'29, committed=1, applied=1))", 2024-04-26T04:32:45.072 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:40.944063+0000", 2024-04-26T04:32:45.072 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 4.0369938239999996, 2024-04-26T04:32:45.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.29890408000000002, 2024-04-26T04:32:45.072 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.072 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.072 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.944063+0000", 2024-04-26T04:32:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.944063+0000", 2024-04-26T04:32:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.944064+0000", 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3969999999999999e-06 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.944071+0000", 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0929999999999997e-06 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.944089+0000", 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8227e-05 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.944101+0000", 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1387e-05 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.242943+0000", 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.29884250400000001 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.242967+0000", 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3472000000000002e-05 2024-04-26T04:32:45.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 47/30 ECSubWriteReply(tid=41, last_complete=46'29, committed=1, applied=1))", 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:41.346876+0000", 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 3.6341808650000003, 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.25977914600000002, 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.346876+0000", 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.346876+0000", 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.346877+0000", 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.11e-06 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.346883+0000", 2024-04-26T04:32:45.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7429999999999999e-06 2024-04-26T04:32:45.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.346903+0000", 2024-04-26T04:32:45.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0727e-05 2024-04-26T04:32:45.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.447030+0000", 2024-04-26T04:32:45.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100126439 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.606593+0000", 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.15956347300000001 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:41.606655+0000", 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1654000000000003e-05 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.079 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.080 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:45.081 INFO:tasks.rados.rados.0.smithi178.stdout:169: finishing write tid 1 to smithi17829308-48 2024-04-26T04:32:45.081 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T04:32:45.082 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:45.076+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:45.182 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:45.176+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:45.251 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:32:45.251 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-04-26T04:32:45.251 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:45.264 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T04:32:45.283 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:45.281+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:45.327 INFO:tasks.rados.rados.0.smithi178.stdout:169: finishing write tid 2 to smithi17829308-48 2024-04-26T04:32:45.327 INFO:tasks.rados.rados.0.smithi178.stdout:169: finishing write tid 3 to smithi17829308-48 2024-04-26T04:32:45.327 INFO:tasks.rados.rados.0.smithi178.stdout:169: finishing write tid 4 to smithi17829308-48 2024-04-26T04:32:45.383 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:45.381+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:45.387 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.7 to 1 (10000) 2024-04-26T04:32:45.394 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:32:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-04-26T04:32:45.394 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-04-26T04:32:45.394 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:45.394 INFO:tasks.rados.rados.0.smithi178.stdout:165: done (2 left) 2024-04-26T04:32:45.394 INFO:tasks.rados.rados.0.smithi178.stdout:170: done (1 left) 2024-04-26T04:32:45.394 INFO:tasks.rados.rados.0.smithi178.stdout:171: snap_remove snap 7 2024-04-26T04:32:45.410 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T04:32:45.411 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-04-26T04:32:45.484 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:45.484+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:45.574 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:45.574 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-04-26T04:32:45.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-04-26T04:32:45.575 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:32:45.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.575 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.8s2 32/30 ECSubWrite(tid=0, reqid=client.4670.0:1, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:45.575 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.200207+0000", 2024-04-26T04:32:45.575 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.370489612, 2024-04-26T04:32:45.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20152414199999999, 2024-04-26T04:32:45.576 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.576 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.576 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200207+0000", 2024-04-26T04:32:45.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200207+0000", 2024-04-26T04:32:45.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200207+0000", 2024-04-26T04:32:45.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6000000000000004e-07 2024-04-26T04:32:45.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200301+0000", 2024-04-26T04:32:45.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4094999999999999e-05 2024-04-26T04:32:45.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200346+0000", 2024-04-26T04:32:45.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5096999999999999e-05 2024-04-26T04:32:45.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.200374+0000", 2024-04-26T04:32:45.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7321000000000002e-05 2024-04-26T04:32:45.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.300611+0000", 2024-04-26T04:32:45.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10023727 2024-04-26T04:32:45.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.300624+0000", 2024-04-26T04:32:45.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3145999999999999e-05 2024-04-26T04:32:45.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.401621+0000", 2024-04-26T04:32:45.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100996801 2024-04-26T04:32:45.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.401731+0000", 2024-04-26T04:32:45.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000109852 2024-04-26T04:32:45.584 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.584 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.584 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.585 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.8s2 32/30 ECSubWrite(tid=1, reqid=client.4670.0:2, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:45.585 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.201963+0000", 2024-04-26T04:32:45.585 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.368733057, 2024-04-26T04:32:45.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.337032781, 2024-04-26T04:32:45.585 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.585 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.586 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.586 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.201963+0000", 2024-04-26T04:32:45.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.201963+0000", 2024-04-26T04:32:45.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.201965+0000", 2024-04-26T04:32:45.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9369999999999998e-06 2024-04-26T04:32:45.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.202008+0000", 2024-04-26T04:32:45.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2861000000000002e-05 2024-04-26T04:32:45.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.202055+0000", 2024-04-26T04:32:45.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6623999999999999e-05 2024-04-26T04:32:45.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.202077+0000", 2024-04-26T04:32:45.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2362999999999999e-05 2024-04-26T04:32:45.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.401252+0000", 2024-04-26T04:32:45.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.19917501200000001 2024-04-26T04:32:45.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.401272+0000", 2024-04-26T04:32:45.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9559999999999999e-05 2024-04-26T04:32:45.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.538951+0000", 2024-04-26T04:32:45.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.137679788 2024-04-26T04:32:45.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.592 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.538996+0000", 2024-04-26T04:32:45.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4635999999999998e-05 2024-04-26T04:32:45.593 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.593 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.593 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.594 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.8s2 32/30 ECSubWrite(tid=2, reqid=client.4670.0:13, at_version=32'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:45.594 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.204365+0000", 2024-04-26T04:32:45.594 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.366331716000001, 2024-04-26T04:32:45.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.33466354999999998, 2024-04-26T04:32:45.594 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.594 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.594 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.594 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.204365+0000", 2024-04-26T04:32:45.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.204365+0000", 2024-04-26T04:32:45.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.204366+0000", 2024-04-26T04:32:45.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7799999999999999e-06 2024-04-26T04:32:45.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.204463+0000", 2024-04-26T04:32:45.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7011999999999999e-05 2024-04-26T04:32:45.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.204508+0000", 2024-04-26T04:32:45.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4953000000000001e-05 2024-04-26T04:32:45.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.304601+0000", 2024-04-26T04:32:45.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100093188 2024-04-26T04:32:45.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.401824+0000", 2024-04-26T04:32:45.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.097222188000000001 2024-04-26T04:32:45.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.401840+0000", 2024-04-26T04:32:45.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5976000000000001e-05 2024-04-26T04:32:45.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.539001+0000", 2024-04-26T04:32:45.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.137161229 2024-04-26T04:32:45.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.539028+0000", 2024-04-26T04:32:45.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7223999999999999e-05 2024-04-26T04:32:45.601 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.601 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.601 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.602 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.es6 32/30 ECSubWrite(tid=157, reqid=client.4670.0:31, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:45.602 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.311566+0000", 2024-04-26T04:32:45.602 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.259130351, 2024-04-26T04:32:45.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20204892699999999, 2024-04-26T04:32:45.602 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.602 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311566+0000", 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311566+0000", 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311567+0000", 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.367e-06 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311906+0000", 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0003388 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311919+0000", 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3345e-05 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.311926+0000", 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6309999999999997e-06 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.412374+0000", 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10044745300000001 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.412396+0000", 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2626999999999999e-05 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.513576+0000", 2024-04-26T04:32:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101179381 2024-04-26T04:32:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.513615+0000", 2024-04-26T04:32:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9323000000000003e-05 2024-04-26T04:32:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.606 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.8s2 32/30 ECSubWrite(tid=3, reqid=client.4670.0:14, at_version=32'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:25.342415+0000", 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 20.228281014, 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.196658685, 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.342415+0000", 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.342415+0000", 2024-04-26T04:32:45.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.342415+0000", 2024-04-26T04:32:45.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.97e-07 2024-04-26T04:32:45.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.342434+0000", 2024-04-26T04:32:45.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8054999999999999e-05 2024-04-26T04:32:45.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.342444+0000", 2024-04-26T04:32:45.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0506000000000001e-05 2024-04-26T04:32:45.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.342454+0000", 2024-04-26T04:32:45.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5119999999999997e-06 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.538170+0000", 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.19571670299999999 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.538177+0000", 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1339999999999998e-06 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.539034+0000", 2024-04-26T04:32:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00085620300000000004 2024-04-26T04:32:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:25.539074+0000", 2024-04-26T04:32:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0374999999999999e-05 2024-04-26T04:32:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.8s2 35/30 ECSubWrite(tid=34, reqid=client.4670.0:202, at_version=35'24, trim_to=0'0, roll_forward_to=35'22))", 2024-04-26T04:32:45.611 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:29.159088+0000", 2024-04-26T04:32:45.612 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 16.411608688000001, 2024-04-26T04:32:45.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.170808823, 2024-04-26T04:32:45.612 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.612 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.612 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.612 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.159088+0000", 2024-04-26T04:32:45.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.159088+0000", 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.159091+0000", 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1700000000000001e-06 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.159834+0000", 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00074291799999999996 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.159871+0000", 2024-04-26T04:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7434000000000001e-05 2024-04-26T04:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.260007+0000", 2024-04-26T04:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100136223 2024-04-26T04:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.328042+0000", 2024-04-26T04:32:45.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.068034609999999995 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.328051+0000", 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0459999999999994e-06 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.329868+0000", 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018171019999999999 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.329896+0000", 2024-04-26T04:32:45.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.832e-05 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 36/30 ECSubWrite(tid=62, reqid=client.4670.0:236, at_version=36'19, trim_to=0'0, roll_forward_to=35'17))", 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:29.632199+0000", 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.938497007, 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.102249196, 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.632199+0000", 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.632199+0000", 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.632201+0000", 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9870000000000002e-06 2024-04-26T04:32:45.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.632244+0000", 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3226999999999998e-05 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.632277+0000", 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2233999999999997e-05 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.732435+0000", 2024-04-26T04:32:45.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100158439 2024-04-26T04:32:45.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.732600+0000", 2024-04-26T04:32:45.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016479900000000001 2024-04-26T04:32:45.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.732615+0000", 2024-04-26T04:32:45.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4993e-05 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.734347+0000", 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0017321330000000001 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.734448+0000", 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000101384 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.8s2 36/30 ECSubWrite(tid=41, reqid=client.4670.0:249, at_version=36'27, trim_to=0'0, roll_forward_to=35'24))", 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:30.074415+0000", 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.496281724999999, 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20176026, 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.074415+0000", 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.074415+0000", 2024-04-26T04:32:45.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.074416+0000", 2024-04-26T04:32:45.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5570000000000001e-06 2024-04-26T04:32:45.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.074459+0000", 2024-04-26T04:32:45.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3106999999999999e-05 2024-04-26T04:32:45.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.074497+0000", 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7939999999999999e-05 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.074521+0000", 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3550999999999999e-05 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.274849+0000", 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20032822 2024-04-26T04:32:45.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.274864+0000", 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4790000000000001e-05 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.276085+0000", 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001220837 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.276175+0000", 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0258000000000002e-05 2024-04-26T04:32:45.624 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 36/30 ECSubWrite(tid=68, reqid=client.4670.0:255, at_version=36'23, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:30.148931+0000", 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.421764920999999, 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.30476068899999997, 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.148931+0000", 2024-04-26T04:32:45.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.148931+0000", 2024-04-26T04:32:45.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.148933+0000", 2024-04-26T04:32:45.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9429999999999999e-06 2024-04-26T04:32:45.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.149084+0000", 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000151039 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.149117+0000", 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2296999999999999e-05 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249245+0000", 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100128513 2024-04-26T04:32:45.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249602+0000", 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000356917 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249618+0000", 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6120000000000002e-05 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.453635+0000", 2024-04-26T04:32:45.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.204016473 2024-04-26T04:32:45.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.453692+0000", 2024-04-26T04:32:45.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7386999999999998e-05 2024-04-26T04:32:45.629 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.629 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.629 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 36/30 ECSubWrite(tid=69, reqid=client.4670.0:256, at_version=36'24, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:30.249317+0000", 2024-04-26T04:32:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.321378774999999, 2024-04-26T04:32:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20442656000000001, 2024-04-26T04:32:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.630 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249317+0000", 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249317+0000", 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249319+0000", 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5230000000000001e-06 2024-04-26T04:32:45.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249437+0000", 2024-04-26T04:32:45.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000118015 2024-04-26T04:32:45.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249464+0000", 2024-04-26T04:32:45.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.745e-05 2024-04-26T04:32:45.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249486+0000", 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1656999999999999e-05 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.349487+0000", 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10000044800000001 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.349501+0000", 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4623999999999999e-05 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.453708+0000", 2024-04-26T04:32:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.104206536 2024-04-26T04:32:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.453744+0000", 2024-04-26T04:32:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6307000000000003e-05 2024-04-26T04:32:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 36/30 ECSubWrite(tid=70, reqid=client.4670.0:257, at_version=36'25, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:30.249529+0000", 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.321166976000001, 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.204255188, 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249529+0000", 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249529+0000", 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249530+0000", 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.7000000000000003e-07 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249562+0000", 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.137e-05 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249586+0000", 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3873999999999999e-05 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.249605+0000", 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9806999999999999e-05 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.450717+0000", 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20111157199999999 2024-04-26T04:32:45.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.450733+0000", 2024-04-26T04:32:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6154000000000002e-05 2024-04-26T04:32:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.453754+0000", 2024-04-26T04:32:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0030204680000000001 2024-04-26T04:32:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.453784+0000", 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0973000000000003e-05 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 36/30 ECSubWrite(tid=73, reqid=client.4670.0:259, at_version=36'27, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:30.255676+0000", 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.315019797, 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.19814728700000001, 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.255676+0000", 2024-04-26T04:32:45.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.255676+0000", 2024-04-26T04:32:45.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.255678+0000", 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7770000000000001e-06 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.255944+0000", 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026597300000000002 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.255975+0000", 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0519999999999999e-05 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.356107+0000", 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100132364 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.451209+0000", 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.095102236000000007 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.451242+0000", 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2499999999999997e-05 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.453793+0000", 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0025512130000000001 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.453824+0000", 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0704000000000003e-05 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 36/30 ECSubWrite(tid=78, reqid=client.4670.0:261, at_version=36'29, trim_to=0'0, roll_forward_to=36'22))", 2024-04-26T04:32:45.643 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:30.457076+0000", 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.113620584, 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.102852337, 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.457076+0000", 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.457076+0000", 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.457078+0000", 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9759999999999998e-06 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.457122+0000", 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4335000000000002e-05 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.457179+0000", 2024-04-26T04:32:45.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6660000000000003e-05 2024-04-26T04:32:45.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.557286+0000", 2024-04-26T04:32:45.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100106906 2024-04-26T04:32:45.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.557788+0000", 2024-04-26T04:32:45.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00050277500000000001 2024-04-26T04:32:45.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.557802+0000", 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4034e-05 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.559870+0000", 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0020680300000000002 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.559928+0000", 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7621000000000003e-05 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.647 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.648 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.648 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 36/30 ECSubWrite(tid=79, reqid=client.4670.0:251, at_version=36'30, trim_to=0'0, roll_forward_to=36'22))", 2024-04-26T04:32:45.648 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:30.557328+0000", 2024-04-26T04:32:45.648 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 15.013367877, 2024-04-26T04:32:45.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.102099829, 2024-04-26T04:32:45.648 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.648 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.648 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.648 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.557328+0000", 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.557328+0000", 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.557329+0000", 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.9999999999999995e-07 2024-04-26T04:32:45.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.557608+0000", 2024-04-26T04:32:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027868700000000001 2024-04-26T04:32:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.557634+0000", 2024-04-26T04:32:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5780000000000001e-05 2024-04-26T04:32:45.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.557647+0000", 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2952999999999999e-05 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.657561+0000", 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.099914113999999998 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.657579+0000", 2024-04-26T04:32:45.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7821000000000001e-05 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.659336+0000", 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0017570330000000001 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.659428+0000", 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2441000000000005e-05 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 36/30 ECSubWrite(tid=84, reqid=client.4670.0:251, at_version=0'0, trim_to=0'0, roll_forward_to=36'30))", 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:30.762759+0000", 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 14.807937179, 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101662381, 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.762759+0000", 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.762759+0000", 2024-04-26T04:32:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.762786+0000", 2024-04-26T04:32:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7134000000000001e-05 2024-04-26T04:32:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.762808+0000", 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1699999999999999e-05 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.762840+0000", 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1826999999999999e-05 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.762855+0000", 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5656999999999999e-05 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.863065+0000", 2024-04-26T04:32:45.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10020961 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.863079+0000", 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4004e-05 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.864350+0000", 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001270731 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.864421+0000", 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1717999999999993e-05 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 40/30 ECSubWrite(tid=105, reqid=client.4670.0:331, at_version=40'34, trim_to=0'0, roll_forward_to=38'33))", 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:34.609812+0000", 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 10.960884126, 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.102068231, 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.609812+0000", 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.609812+0000", 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.609814+0000", 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6500000000000001e-06 2024-04-26T04:32:45.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.609857+0000", 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.2823e-05 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.609885+0000", 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8804000000000001e-05 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.710013+0000", 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100127306 2024-04-26T04:32:45.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.710175+0000", 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016273500000000001 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.710203+0000", 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7551000000000001e-05 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.711785+0000", 2024-04-26T04:32:45.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001582434 2024-04-26T04:32:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.711880+0000", 2024-04-26T04:32:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.4927999999999996e-05 2024-04-26T04:32:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.6s4 40/30 ECSubWrite(tid=16, reqid=client.4670.0:339, at_version=40'9, trim_to=0'0, roll_forward_to=40'7))", 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.170897+0000", 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 10.399799587, 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10373853099999999, 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.170897+0000", 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.170897+0000", 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.170899+0000", 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.773e-06 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.171076+0000", 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017662400000000001 2024-04-26T04:32:45.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.171127+0000", 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0467000000000003e-05 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.271271+0000", 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10014450599999999 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.271445+0000", 2024-04-26T04:32:45.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000174148 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.271482+0000", 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6647000000000003e-05 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.274552+0000", 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0030700499999999999 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.274635+0000", 2024-04-26T04:32:45.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3316e-05 2024-04-26T04:32:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 41/30 ECSubWrite(tid=115, reqid=client.4670.0:355, at_version=41'36, trim_to=0'0, roll_forward_to=40'34))", 2024-04-26T04:32:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.542214+0000", 2024-04-26T04:32:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 10.02848191, 2024-04-26T04:32:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10409394499999999, 2024-04-26T04:32:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.666 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.542214+0000", 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.542214+0000", 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.542217+0000", 2024-04-26T04:32:45.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2220000000000001e-06 2024-04-26T04:32:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.542406+0000", 2024-04-26T04:32:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018903600000000001 2024-04-26T04:32:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.542453+0000", 2024-04-26T04:32:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.7274000000000002e-05 2024-04-26T04:32:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.542472+0000", 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9522999999999999e-05 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.642685+0000", 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100212939 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.642713+0000", 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8021000000000002e-05 2024-04-26T04:32:45.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.646220+0000", 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003506285 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.646308+0000", 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8645000000000003e-05 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.1s5 41/30 ECSubWrite(tid=116, reqid=client.4670.0:356, at_version=41'37, trim_to=0'0, roll_forward_to=40'34))", 2024-04-26T04:32:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.542532+0000", 2024-04-26T04:32:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 10.028164711000001, 2024-04-26T04:32:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.104871257, 2024-04-26T04:32:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.542532+0000", 2024-04-26T04:32:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.542532+0000", 2024-04-26T04:32:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.542533+0000", 2024-04-26T04:32:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.573e-06 2024-04-26T04:32:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.542679+0000", 2024-04-26T04:32:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000145678 2024-04-26T04:32:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.542702+0000", 2024-04-26T04:32:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.304e-05 2024-04-26T04:32:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.642814+0000", 2024-04-26T04:32:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100112062 2024-04-26T04:32:45.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.644353+0000", 2024-04-26T04:32:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0015390619999999999 2024-04-26T04:32:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.644379+0000", 2024-04-26T04:32:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5550000000000001e-05 2024-04-26T04:32:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.647211+0000", 2024-04-26T04:32:45.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0028321370000000002 2024-04-26T04:32:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.647403+0000", 2024-04-26T04:32:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00019215500000000001 2024-04-26T04:32:45.678 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.bs3 41/30 ECSubWrite(tid=120, reqid=client.4670.0:344, at_version=41'21, trim_to=0'0, roll_forward_to=40'20))", 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.747687+0000", 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 9.8230088040000005, 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10201139400000001, 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:45.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.747687+0000", 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.747687+0000", 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.747690+0000", 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7e-06 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.747800+0000", 2024-04-26T04:32:45.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000109924 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.747844+0000", 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.3454000000000001e-05 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.747879+0000", 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5873e-05 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.848126+0000", 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10024613 2024-04-26T04:32:45.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.848142+0000", 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6889999999999999e-05 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.849647+0000", 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0015047929999999999 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.849699+0000", 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1629999999999999e-05 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.682 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:45.683 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:45.683 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:45.684 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T04:32:45.685 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:45.680+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:45.785 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:45.782+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:45.815 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:45.815 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 0 2024-04-26T04:32:45.815 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:45.829 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T04:32:45.886 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:45.884+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:45.946 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:45.946 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:45.947 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:32:45.947 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:32:45.947 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:45.961 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T04:32:45.987 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:45.985+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:46.088 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:46.088 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:32:46.088 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:32:46.088 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:32:46.088 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.088 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 35/30 ECSubWrite(tid=28, reqid=client.4670.0:219, at_version=35'21, trim_to=0'0, roll_forward_to=34'14))", 2024-04-26T04:32:46.088 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.554372+0000", 2024-04-26T04:32:46.088 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 17.531483740999999, 2024-04-26T04:32:46.088 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10197365, 2024-04-26T04:32:46.088 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554372+0000", 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554372+0000", 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.089 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.090 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.090 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554373+0000", 2024-04-26T04:32:46.090 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.063e-06 2024-04-26T04:32:46.090 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.090 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.090 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.090 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554429+0000", 2024-04-26T04:32:46.090 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.6147000000000003e-05 2024-04-26T04:32:46.090 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.090 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.090 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.090 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554447+0000", 2024-04-26T04:32:46.090 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8159e-05 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.554537+0000", 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.9988000000000006e-05 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.654747+0000", 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10020931199999999 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.091 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.654776+0000", 2024-04-26T04:32:46.092 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9085e-05 2024-04-26T04:32:46.092 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.092 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.092 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.092 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.656276+0000", 2024-04-26T04:32:46.092 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001500237 2024-04-26T04:32:46.092 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.092 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.092 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.092 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.656346+0000", 2024-04-26T04:32:46.092 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.9659000000000005e-05 2024-04-26T04:32:46.092 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.092 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.8s3 35/30 ECSubRead(tid=30, to_read={3:148e36d6:::smithi17829308-35:head=0,143360,0}, subchunks={3:148e36d6:::smithi17829308-35:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.660368+0000", 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 17.425487515, 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10139301100000001, 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.660368+0000", 2024-04-26T04:32:46.093 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.094 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.094 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.094 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.094 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.660368+0000", 2024-04-26T04:32:46.094 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.094 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.094 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.094 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.094 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.660370+0000", 2024-04-26T04:32:46.094 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.606e-06 2024-04-26T04:32:46.094 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.094 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.094 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.660388+0000", 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8586000000000001e-05 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.660454+0000", 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.5843e-05 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.660473+0000", 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8552000000000001e-05 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.095 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.760802+0000", 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100329317 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.761761+0000", 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00095910699999999995 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 35/30 ECSubWrite(tid=33, reqid=client.4670.0:220, at_version=0'0, trim_to=0'0, roll_forward_to=35'22))", 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.867463+0000", 2024-04-26T04:32:46.096 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 17.218392938000001, 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10178548499999999, 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.867463+0000", 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.867463+0000", 2024-04-26T04:32:46.097 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.867465+0000", 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.514e-06 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.867496+0000", 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0574000000000002e-05 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.098 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.867527+0000", 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1015e-05 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.967671+0000", 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100144222 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.967831+0000", 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016030100000000001 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.099 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.967847+0000", 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5415999999999999e-05 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.969178+0000", 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00133096 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.969248+0000", 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0482999999999996e-05 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.100 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 35/30 ECSubWrite(tid=34, reqid=client.4670.0:202, at_version=35'24, trim_to=0'0, roll_forward_to=35'22))", 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.161613+0000", 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.924242706000001, 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.103291546, 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.101 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.161613+0000", 2024-04-26T04:32:46.102 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.102 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.102 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.102 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.102 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.161613+0000", 2024-04-26T04:32:46.102 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.102 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.102 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.102 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.102 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.161615+0000", 2024-04-26T04:32:46.102 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.204e-06 2024-04-26T04:32:46.102 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.102 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.162484+0000", 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00086872700000000002 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.162526+0000", 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1791e-05 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.162550+0000", 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4341999999999998e-05 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.103 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.262786+0000", 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100236265 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.262801+0000", 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5012e-05 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.264812+0000", 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.002010333 2024-04-26T04:32:46.104 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.264905+0000", 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.2872000000000001e-05 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs2 36/30 ECSubWrite(tid=80, reqid=client.4670.0:253, at_version=0'0, trim_to=0'0, roll_forward_to=36'13))", 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.410070+0000", 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 15.675786063, 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10175970300000001, 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.105 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.106 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.106 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.106 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.106 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.106 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.410070+0000", 2024-04-26T04:32:46.106 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.106 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.106 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.106 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.106 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.410070+0000", 2024-04-26T04:32:46.106 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.106 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.106 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.410072+0000", 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0729999999999999e-06 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.410103+0000", 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0725000000000001e-05 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.410148+0000", 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.5757999999999998e-05 2024-04-26T04:32:46.107 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.510252+0000", 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10010348099999999 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.510407+0000", 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015535200000000001 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.510435+0000", 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8252e-05 2024-04-26T04:32:46.108 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.511793+0000", 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001357659 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.511829+0000", 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.6402999999999997e-05 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.109 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs2 36/30 ECSubWrite(tid=85, reqid=client.4670.0:277, at_version=36'14, trim_to=0'0, roll_forward_to=36'13))", 2024-04-26T04:32:46.110 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.967458+0000", 2024-04-26T04:32:46.110 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 15.118397597, 2024-04-26T04:32:46.110 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101510735, 2024-04-26T04:32:46.110 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.110 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.110 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.110 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.110 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.110 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.110 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.967458+0000", 2024-04-26T04:32:46.110 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.110 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.110 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.967458+0000", 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.967460+0000", 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.283e-06 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.967547+0000", 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6652999999999999e-05 2024-04-26T04:32:46.111 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.967579+0000", 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1538000000000002e-05 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:31.067702+0000", 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100123669 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:31.067824+0000", 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000121827 2024-04-26T04:32:46.112 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:31.067834+0000", 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.5279999999999992e-06 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:31.068887+0000", 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0010531850000000001 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:31.068969+0000", 2024-04-26T04:32:46.113 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2052000000000007e-05 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 40/30 ECSubWrite(tid=26, reqid=osd.8.0:26, at_version=40'16, trim_to=0'0, roll_forward_to=39'14))", 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:34.621563+0000", 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 11.464292576, 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10188129999999999, 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.114 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:34.621563+0000", 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:34.621563+0000", 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:34.621564+0000", 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.4300000000000003e-07 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.115 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.116 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.116 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:34.621695+0000", 2024-04-26T04:32:46.116 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013113400000000001 2024-04-26T04:32:46.116 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.116 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.116 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.116 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:34.621719+0000", 2024-04-26T04:32:46.116 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3788999999999999e-05 2024-04-26T04:32:46.116 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.116 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.116 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.116 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:34.621734+0000", 2024-04-26T04:32:46.116 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4849e-05 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:34.721973+0000", 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10023960799999999 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:34.721989+0000", 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5546e-05 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.117 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:34.723361+0000", 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001371812 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:34.723445+0000", 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4018999999999996e-05 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.6s2 40/30 ECSubWrite(tid=16, reqid=client.4670.0:339, at_version=40'9, trim_to=0'0, roll_forward_to=40'7))", 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.172685+0000", 2024-04-26T04:32:46.118 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 10.913170913, 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102942541, 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.172685+0000", 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.172685+0000", 2024-04-26T04:32:46.119 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.172687+0000", 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.784e-06 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.173214+0000", 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00052737900000000002 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.173243+0000", 2024-04-26T04:32:46.120 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9391999999999999e-05 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.273386+0000", 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10014255399999999 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.273586+0000", 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000199731 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.121 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.273605+0000", 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9264999999999999e-05 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.275566+0000", 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0019612660000000001 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.275627+0000", 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.1169999999999999e-05 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.122 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.6s2 40/30 ECSubWrite(tid=17, reqid=client.4670.0:340, at_version=40'10, trim_to=0'0, roll_forward_to=40'7))", 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.273438+0000", 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 10.812417973000001, 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10222980600000001, 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.273438+0000", 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.123 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.273438+0000", 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.273439+0000", 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.04e-06 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.273680+0000", 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024102600000000001 2024-04-26T04:32:46.124 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.273706+0000", 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6449e-05 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.373880+0000", 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10017335500000001 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.374060+0000", 2024-04-26T04:32:46.125 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018035600000000001 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.374093+0000", 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2804e-05 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.375606+0000", 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0015129189999999999 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.126 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.375668+0000", 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.1857000000000001e-05 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.6s2 40/30 ECSubWrite(tid=19, reqid=client.4670.0:342, at_version=40'12, trim_to=0'0, roll_forward_to=40'7))", 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.374141+0000", 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 10.711714345000001, 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10140653600000001, 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.127 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.128 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.128 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.128 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.374141+0000", 2024-04-26T04:32:46.128 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.128 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.128 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.128 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.128 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.374141+0000", 2024-04-26T04:32:46.128 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.128 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.128 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.128 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.129 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.374142+0000", 2024-04-26T04:32:46.129 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7699999999999992e-07 2024-04-26T04:32:46.129 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.129 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.129 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.129 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.374178+0000", 2024-04-26T04:32:46.129 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.6015e-05 2024-04-26T04:32:46.129 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.129 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.129 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.129 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.374201+0000", 2024-04-26T04:32:46.129 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2161999999999999e-05 2024-04-26T04:32:46.130 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.130 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.130 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.130 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.474301+0000", 2024-04-26T04:32:46.130 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100100781 2024-04-26T04:32:46.130 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.130 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.130 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.130 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.474477+0000", 2024-04-26T04:32:46.130 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017601499999999999 2024-04-26T04:32:46.130 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.130 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.130 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.474493+0000", 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5432999999999999e-05 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.475490+0000", 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00099722900000000008 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.475548+0000", 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.7924000000000003e-05 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.131 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs2 41/30 ECSubWrite(tid=121, reqid=client.4670.0:344, at_version=0'0, trim_to=0'0, roll_forward_to=41'21))", 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.952311+0000", 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 10.133544778999999, 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101772101, 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.132 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.952311+0000", 2024-04-26T04:32:46.133 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.133 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.133 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.133 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.133 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.952311+0000", 2024-04-26T04:32:46.133 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.133 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.133 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.133 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.133 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.952313+0000", 2024-04-26T04:32:46.133 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.643e-06 2024-04-26T04:32:46.133 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.133 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.952334+0000", 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1826000000000001e-05 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.952366+0000", 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1251999999999998e-05 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:36.052472+0000", 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10010649100000001 2024-04-26T04:32:46.134 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:36.052655+0000", 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000182979 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:36.052680+0000", 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4709e-05 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:36.053972+0000", 2024-04-26T04:32:46.135 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0012924259999999999 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:36.054083+0000", 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000110775 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 45/30 ECSubWrite(tid=31, reqid=client.4670.0:374, at_version=45'20, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.532047+0000", 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.5538091999999999, 2024-04-26T04:32:46.136 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17995593700000001, 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532047+0000", 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532047+0000", 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.137 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.138 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.138 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532048+0000", 2024-04-26T04:32:46.138 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5960000000000001e-06 2024-04-26T04:32:46.138 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.138 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.138 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.138 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532346+0000", 2024-04-26T04:32:46.138 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00029776800000000001 2024-04-26T04:32:46.138 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.138 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.138 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.138 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532375+0000", 2024-04-26T04:32:46.138 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8915e-05 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532395+0000", 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9663000000000001e-05 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608323+0000", 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.075928566000000003 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608336+0000", 2024-04-26T04:32:46.139 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3377000000000001e-05 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.711932+0000", 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10359542099999999 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.712003+0000", 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0630999999999995e-05 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.140 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 45/30 ECSubWrite(tid=32, reqid=client.4670.0:375, at_version=45'21, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.533950+0000", 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.5519055100000001, 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17809028800000001, 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.533950+0000", 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.141 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.533950+0000", 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.533951+0000", 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1200000000000002e-07 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.534075+0000", 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012397999999999999 2024-04-26T04:32:46.142 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.534089+0000", 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4097000000000001e-05 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.534097+0000", 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.6420000000000007e-06 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.709588+0000", 2024-04-26T04:32:46.143 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17549146800000001 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.709604+0000", 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5400000000000002e-05 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.712011+0000", 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0024075469999999999 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.144 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.712041+0000", 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9342000000000001e-05 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 45/30 ECSubWrite(tid=33, reqid=client.4670.0:376, at_version=45'22, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.535418+0000", 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.5504378599999997, 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.177833458, 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.145 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535418+0000", 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535418+0000", 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535419+0000", 2024-04-26T04:32:46.146 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6400000000000001e-07 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535502+0000", 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.365e-05 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535517+0000", 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4489e-05 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535525+0000", 2024-04-26T04:32:46.147 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4940000000000003e-06 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.710429+0000", 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17490356100000001 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.710454+0000", 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5038999999999999e-05 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.148 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.713205+0000", 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00275055 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.713251+0000", 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.6811000000000003e-05 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 45/30 ECSubWrite(tid=34, reqid=client.4670.0:377, at_version=45'23, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.536571+0000", 2024-04-26T04:32:46.149 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.5492843050000005, 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17941110099999999, 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536571+0000", 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536571+0000", 2024-04-26T04:32:46.150 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536572+0000", 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.0999999999999998e-07 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536589+0000", 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7110000000000001e-05 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536601+0000", 2024-04-26T04:32:46.151 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1368e-05 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536608+0000", 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2490000000000004e-06 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.711524+0000", 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17491638100000001 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.152 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.711553+0000", 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9142e-05 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.715942+0000", 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00438895 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.715983+0000", 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.0290999999999998e-05 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.153 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.as2 46/30 ECSubWrite(tid=35, reqid=client.4670.0:387, at_version=46'27, trim_to=0'0, roll_forward_to=44'25))", 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:40.475172+0000", 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.6106842690000001, 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10241877200000001, 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.475172+0000", 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.154 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.475172+0000", 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.475172+0000", 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.8999999999999996e-07 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.475223+0000", 2024-04-26T04:32:46.155 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.1034e-05 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.475251+0000", 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7733e-05 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.475274+0000", 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3578999999999999e-05 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.156 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.575517+0000", 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100242492 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.575533+0000", 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5594e-05 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.577506+0000", 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0019738899999999998 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.157 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.158 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.577590+0000", 2024-04-26T04:32:46.158 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.386e-05 2024-04-26T04:32:46.158 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.158 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.158 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.158 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.158 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.158 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 46/30 ECSubWrite(tid=39, reqid=osd.8.0:39, at_version=46'27, trim_to=0'0, roll_forward_to=45'25))", 2024-04-26T04:32:46.158 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:40.841568+0000", 2024-04-26T04:32:46.158 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.244287409, 2024-04-26T04:32:46.158 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102199524, 2024-04-26T04:32:46.158 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.158 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.841568+0000", 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.841568+0000", 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.159 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.841571+0000", 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2029999999999999e-06 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.841625+0000", 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.4827999999999998e-05 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.841667+0000", 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1152000000000002e-05 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.160 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.161 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.941807+0000", 2024-04-26T04:32:46.161 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10014062999999999 2024-04-26T04:32:46.161 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.161 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.161 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.161 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.941994+0000", 2024-04-26T04:32:46.161 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018704600000000001 2024-04-26T04:32:46.161 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.161 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.161 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.161 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.942021+0000", 2024-04-26T04:32:46.161 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6438999999999999e-05 2024-04-26T04:32:46.161 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.162 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.162 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.162 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.943678+0000", 2024-04-26T04:32:46.162 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001657457 2024-04-26T04:32:46.162 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.162 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.162 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.162 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.943768+0000", 2024-04-26T04:32:46.162 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.9769000000000001e-05 2024-04-26T04:32:46.162 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.162 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.162 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.162 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 47/30 ECSubWrite(tid=55, reqid=osd.5.0:55, at_version=47'43, trim_to=0'0, roll_forward_to=44'41))", 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:41.211282+0000", 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.8745736559999999, 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.25277038499999999, 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211282+0000", 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.163 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211282+0000", 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211283+0000", 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6400000000000002e-07 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211337+0000", 2024-04-26T04:32:46.164 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.4304e-05 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211361+0000", 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3983000000000001e-05 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211395+0000", 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4255000000000003e-05 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.363290+0000", 2024-04-26T04:32:46.165 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.15189467800000001 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.363307+0000", 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6863000000000001e-05 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.463984+0000", 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100677144 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.166 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.464053+0000", 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.8493999999999998e-05 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 47/30 ECSubWrite(tid=41, reqid=osd.8.0:40, at_version=0'0, trim_to=0'0, roll_forward_to=46'29))", 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:41.246160+0000", 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.8396955889999997, 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10221015, 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.167 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.246160+0000", 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.246160+0000", 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.246161+0000", 2024-04-26T04:32:46.168 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1599999999999999e-06 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.246186+0000", 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4528999999999998e-05 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.246216+0000", 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0148999999999999e-05 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.169 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.346316+0000", 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10009957 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.346467+0000", 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015141500000000001 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.346492+0000", 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4964999999999999e-05 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.170 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348290+0000", 2024-04-26T04:32:46.171 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0017975949999999999 2024-04-26T04:32:46.171 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.171 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.171 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.171 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348370+0000", 2024-04-26T04:32:46.171 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0766999999999994e-05 2024-04-26T04:32:46.171 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.171 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.171 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.171 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.171 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.171 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "pg_update_log_missing(3.8s3 epoch 50/30 rep_tid 66 entries 50'54 (0'0) error 3:1d1feae2:::smithi17829308-1:head by client.4670.0:422 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 50'51)", 2024-04-26T04:32:46.171 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:44.585396+0000", 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.5004598850000002, 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20079079399999999, 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585396+0000", 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.172 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585396+0000", 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585398+0000", 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7999999999999999e-06 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585418+0000", 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0358999999999999e-05 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.173 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585468+0000", 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.9444999999999998e-05 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.685596+0000", 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100128605 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.685857+0000", 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026076799999999998 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.174 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.175 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.786187+0000", 2024-04-26T04:32:46.175 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100329817 2024-04-26T04:32:46.175 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.175 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.175 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.175 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.175 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.175 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:46.176 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:46.085+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:46.177 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.8 to 1 (10000) 2024-04-26T04:32:46.177 INFO:tasks.rados.rados.0.smithi178.stdout:171: done (1 left) 2024-04-26T04:32:46.177 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 6 current snap is 9 2024-04-26T04:32:46.177 INFO:tasks.rados.rados.0.smithi178.stdout:169: finishing write tid 5 to smithi17829308-48 2024-04-26T04:32:46.177 INFO:tasks.rados.rados.0.smithi178.stdout:169: oid 48 updating version 0 to 35 2024-04-26T04:32:46.177 INFO:tasks.rados.rados.0.smithi178.stdout:169: oid 48 version 35 is already newer than 33 2024-04-26T04:32:46.177 INFO:tasks.rados.rados.0.smithi178.stdout:169: oid 48 version 35 is already newer than 34 2024-04-26T04:32:46.177 INFO:tasks.rados.rados.0.smithi178.stdout:169: oid 48 updating version 35 to 36 2024-04-26T04:32:46.177 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 48 v 36 (ObjNum 81 snap 8 seq_num 81) dirty exists 2024-04-26T04:32:46.178 INFO:tasks.rados.rados.0.smithi178.stdout:169: left oid 48 (ObjNum 81 snap 8 seq_num 81) 2024-04-26T04:32:46.178 INFO:tasks.rados.rados.0.smithi178.stdout:172: seq_num 82 ranges {98304=622592,720896=475136,1196032=294912} 2024-04-26T04:32:46.178 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T04:32:46.181 INFO:tasks.rados.rados.0.smithi178.stdout:172: writing smithi17829308-6 from 98304 to 720896 tid 1 2024-04-26T04:32:46.188 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-04-26T04:32:46.189 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:46.185+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:46.189 INFO:tasks.rados.rados.0.smithi178.stdout:172: writing smithi17829308-6 from 720896 to 1196032 tid 2 2024-04-26T04:32:46.193 INFO:tasks.rados.rados.0.smithi178.stdout:172: writing smithi17829308-6 from 1196032 to 1490944 tid 3 2024-04-26T04:32:46.194 INFO:tasks.rados.rados.0.smithi178.stdout:169: done (1 left) 2024-04-26T04:32:46.194 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 10 current snap is 9 2024-04-26T04:32:46.194 INFO:tasks.rados.rados.0.smithi178.stdout:173: seq_num 83 ranges {638976=638976,1277952=622592,1900544=147456} 2024-04-26T04:32:46.203 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-26T04:32:46.214 INFO:tasks.rados.rados.0.smithi178.stdout:173: writing smithi17829308-10 from 638976 to 1277952 tid 1 2024-04-26T04:32:46.222 INFO:tasks.rados.rados.0.smithi178.stdout:173: writing smithi17829308-10 from 1277952 to 1900544 tid 2 2024-04-26T04:32:46.225 INFO:tasks.rados.rados.0.smithi178.stdout:173: writing smithi17829308-10 from 1900544 to 2048000 tid 3 2024-04-26T04:32:46.225 INFO:tasks.rados.rados.0.smithi178.stdout:174: setattr oid 50 current snap is 9 2024-04-26T04:32:46.229 INFO:tasks.rados.rados.0.smithi178.stdout:175: read oid 31 snap 8 2024-04-26T04:32:46.229 INFO:tasks.rados.rados.0.smithi178.stdout:175: expect (ObjNum 78 snap 8 seq_num 78) 2024-04-26T04:32:46.229 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 2 current snap is 9 2024-04-26T04:32:46.229 INFO:tasks.rados.rados.0.smithi178.stdout:176: seq_num 84 ranges {2244608=655360,2899968=638976,3538944=131072} 2024-04-26T04:32:46.256 INFO:tasks.rados.rados.0.smithi178.stdout:176: writing smithi17829308-2 from 2244608 to 2899968 tid 1 2024-04-26T04:32:46.261 INFO:tasks.rados.rados.0.smithi178.stdout:176: writing smithi17829308-2 from 2899968 to 3538944 tid 2 2024-04-26T04:32:46.262 INFO:tasks.rados.rados.0.smithi178.stdout:176: writing smithi17829308-2 from 3538944 to 3670016 tid 3 2024-04-26T04:32:46.262 INFO:tasks.rados.rados.0.smithi178.stdout:177: snap_create 2024-04-26T04:32:46.289 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:46.284+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:46.307 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:46.307 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:46.307 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 0 2024-04-26T04:32:46.307 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:46.322 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T04:32:46.390 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:46.388+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:46.446 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:46.446 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:46.446 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:32:46.447 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:32:46.447 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:46.461 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T04:32:46.505 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:46.505+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:46.589 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:46.589 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:32:46.589 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:32:46.589 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:32:46.589 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.590 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:1 3.8s0 3:1d1feae2:::smithi17829308-1:head [append 0~32768,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:46.590 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.200828+0000", 2024-04-26T04:32:46.590 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 21.386343102000001, 2024-04-26T04:32:46.590 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.311358421, 2024-04-26T04:32:46.590 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.590 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.590 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:46.590 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:46.590 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:46.590 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:46.590 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 1 2024-04-26T04:32:46.590 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.591 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.591 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.591 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.591 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200828+0000", 2024-04-26T04:32:46.591 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.591 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.591 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.591 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.591 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200828+0000", 2024-04-26T04:32:46.591 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.591 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.591 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.591 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.592 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200830+0000", 2024-04-26T04:32:46.592 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.57e-06 2024-04-26T04:32:46.592 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.592 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.592 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.592 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200962+0000", 2024-04-26T04:32:46.592 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000132115 2024-04-26T04:32:46.592 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.592 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.592 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.592 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200985+0000", 2024-04-26T04:32:46.593 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3065e-05 2024-04-26T04:32:46.593 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.593 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.593 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.593 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.201007+0000", 2024-04-26T04:32:46.593 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1333000000000001e-05 2024-04-26T04:32:46.593 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.593 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.593 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.593 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.201181+0000", 2024-04-26T04:32:46.593 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017403899999999999 2024-04-26T04:32:46.593 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.593 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.594 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:46.594 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.201416+0000", 2024-04-26T04:32:46.594 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000234713 2024-04-26T04:32:46.594 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.594 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.594 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.594 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.202587+0000", 2024-04-26T04:32:46.594 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001171727 2024-04-26T04:32:46.594 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.594 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.594 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.594 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.345497+0000", 2024-04-26T04:32:46.594 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.142909339 2024-04-26T04:32:46.595 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.595 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.595 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:46.595 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.511984+0000", 2024-04-26T04:32:46.595 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.16648707800000001 2024-04-26T04:32:46.595 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.595 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.595 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.595 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.512186+0000", 2024-04-26T04:32:46.595 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000202442 2024-04-26T04:32:46.595 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.595 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.595 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 32/30 ECSubWriteReply(tid=2, last_complete=32'3, committed=1, applied=1))", 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.212591+0000", 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 21.374579568000001, 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.23443240400000001, 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212591+0000", 2024-04-26T04:32:46.596 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212591+0000", 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212593+0000", 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9240000000000001e-06 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.597 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212604+0000", 2024-04-26T04:32:46.598 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1225e-05 2024-04-26T04:32:46.598 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.598 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.598 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.598 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212644+0000", 2024-04-26T04:32:46.598 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9538e-05 2024-04-26T04:32:46.598 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.598 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.599 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.599 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212665+0000", 2024-04-26T04:32:46.599 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1073e-05 2024-04-26T04:32:46.599 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.599 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.599 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.599 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.446985+0000", 2024-04-26T04:32:46.599 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.23432027899999999 2024-04-26T04:32:46.599 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.599 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.599 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.599 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.447024+0000", 2024-04-26T04:32:46.599 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8365000000000003e-05 2024-04-26T04:32:46.600 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.600 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.600 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.600 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.600 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.600 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:244 3.8s0 3:148e36d6:::smithi17829308-35:2 [read 0~573440,checksum in=4b,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:46.600 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.768310+0000", 2024-04-26T04:32:46.600 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.818860470000001, 2024-04-26T04:32:46.600 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.303379758, 2024-04-26T04:32:46.600 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.600 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.600 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:46.600 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 244 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768310+0000", 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768310+0000", 2024-04-26T04:32:46.601 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.602 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.602 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.602 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.602 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768313+0000", 2024-04-26T04:32:46.602 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.779e-06 2024-04-26T04:32:46.602 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.602 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.602 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.603 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768326+0000", 2024-04-26T04:32:46.603 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.258e-05 2024-04-26T04:32:46.603 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.603 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.603 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.603 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768356+0000", 2024-04-26T04:32:46.603 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0620999999999997e-05 2024-04-26T04:32:46.603 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.603 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.603 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.603 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768371+0000", 2024-04-26T04:32:46.603 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5140000000000001e-05 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768568+0000", 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00019688299999999999 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768779+0000", 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021095900000000001 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.604 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071690+0000", 2024-04-26T04:32:46.605 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30291079599999998 2024-04-26T04:32:46.605 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.605 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.605 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.605 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.605 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.605 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:245 3.8s0 3:148e36d6:::smithi17829308-35:2 [read 0~573440,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:46.605 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.768406+0000", 2024-04-26T04:32:46.605 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.818764658999999, 2024-04-26T04:32:46.605 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30361496399999999, 2024-04-26T04:32:46.605 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.605 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.605 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:46.606 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:46.606 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:46.606 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:46.606 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 245 2024-04-26T04:32:46.606 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.606 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.606 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.606 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.606 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768406+0000", 2024-04-26T04:32:46.606 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.606 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.606 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.606 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768406+0000", 2024-04-26T04:32:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768408+0000", 2024-04-26T04:32:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4670000000000001e-06 2024-04-26T04:32:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768417+0000", 2024-04-26T04:32:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3230000000000004e-06 2024-04-26T04:32:46.607 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.608 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.608 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.608 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768459+0000", 2024-04-26T04:32:46.608 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1820999999999998e-05 2024-04-26T04:32:46.608 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.608 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.608 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.608 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768483+0000", 2024-04-26T04:32:46.608 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4307999999999998e-05 2024-04-26T04:32:46.608 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.608 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.609 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.609 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769025+0000", 2024-04-26T04:32:46.609 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000542086 2024-04-26T04:32:46.609 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.609 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.609 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:46.609 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769073+0000", 2024-04-26T04:32:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.7707999999999998e-05 2024-04-26T04:32:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072021+0000", 2024-04-26T04:32:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.302948251 2024-04-26T04:32:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:246 3.8s0 3:148e36d6:::smithi17829308-35:2 [read 0~573440,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:46.610 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.768569+0000", 2024-04-26T04:32:46.611 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.818601764, 2024-04-26T04:32:46.611 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30420794400000001, 2024-04-26T04:32:46.611 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.611 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.611 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:46.611 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:46.611 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:46.611 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:46.611 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 246 2024-04-26T04:32:46.611 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.611 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.611 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.611 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.612 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768569+0000", 2024-04-26T04:32:46.612 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.612 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.612 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.612 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.612 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768569+0000", 2024-04-26T04:32:46.612 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.612 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.612 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.612 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.612 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768570+0000", 2024-04-26T04:32:46.612 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2270000000000001e-06 2024-04-26T04:32:46.613 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.613 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.613 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.613 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768578+0000", 2024-04-26T04:32:46.613 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8520000000000004e-06 2024-04-26T04:32:46.613 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.613 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.613 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.613 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768600+0000", 2024-04-26T04:32:46.613 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1948999999999999e-05 2024-04-26T04:32:46.613 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.613 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.613 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.614 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768618+0000", 2024-04-26T04:32:46.614 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7615999999999999e-05 2024-04-26T04:32:46.614 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.614 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.614 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.614 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769307+0000", 2024-04-26T04:32:46.614 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00068885099999999998 2024-04-26T04:32:46.614 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.614 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.614 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:46.614 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769389+0000", 2024-04-26T04:32:46.614 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2391000000000004e-05 2024-04-26T04:32:46.615 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.615 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.615 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.615 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072777+0000", 2024-04-26T04:32:46.615 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30338805800000002 2024-04-26T04:32:46.615 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.615 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.615 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.615 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.615 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.615 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=36, attrs_read=0))", 2024-04-26T04:32:46.615 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.770066+0000", 2024-04-26T04:32:46.615 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.817104513, 2024-04-26T04:32:46.616 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30163910300000002, 2024-04-26T04:32:46.616 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.616 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.616 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.616 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.616 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.616 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.616 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770066+0000", 2024-04-26T04:32:46.616 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.616 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.616 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.616 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.616 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770066+0000", 2024-04-26T04:32:46.617 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.617 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.617 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.617 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.617 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770067+0000", 2024-04-26T04:32:46.617 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.78e-07 2024-04-26T04:32:46.617 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.617 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.617 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.617 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770288+0000", 2024-04-26T04:32:46.617 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000220583 2024-04-26T04:32:46.617 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.617 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.618 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.618 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770301+0000", 2024-04-26T04:32:46.618 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3219e-05 2024-04-26T04:32:46.618 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.618 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.618 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.618 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870381+0000", 2024-04-26T04:32:46.618 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100080399 2024-04-26T04:32:46.618 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.618 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.618 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.619 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071285+0000", 2024-04-26T04:32:46.619 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20090390799999999 2024-04-26T04:32:46.619 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.619 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.619 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.619 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071705+0000", 2024-04-26T04:32:46.619 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000420116 2024-04-26T04:32:46.619 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.619 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.619 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.619 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.620 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.620 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-26T04:32:46.620 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.770296+0000", 2024-04-26T04:32:46.620 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.816874359, 2024-04-26T04:32:46.620 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30079836799999998, 2024-04-26T04:32:46.620 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.620 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.620 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.620 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.620 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.620 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.620 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770296+0000", 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770296+0000", 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770298+0000", 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.403e-06 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.621 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.622 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770560+0000", 2024-04-26T04:32:46.622 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026199900000000001 2024-04-26T04:32:46.622 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.622 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.622 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.622 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770587+0000", 2024-04-26T04:32:46.622 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7126000000000001e-05 2024-04-26T04:32:46.622 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.622 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.622 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.622 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770605+0000", 2024-04-26T04:32:46.622 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8346e-05 2024-04-26T04:32:46.623 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.623 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.623 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.623 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071054+0000", 2024-04-26T04:32:46.623 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.300448465 2024-04-26T04:32:46.623 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.623 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.623 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.623 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071095+0000", 2024-04-26T04:32:46.623 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1029000000000003e-05 2024-04-26T04:32:46.623 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.623 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.623 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.624 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.624 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.624 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=38, attrs_read=0))", 2024-04-26T04:32:46.624 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.770642+0000", 2024-04-26T04:32:46.624 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.816528400999999, 2024-04-26T04:32:46.624 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30050239499999998, 2024-04-26T04:32:46.624 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.624 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.624 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.624 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.624 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.624 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.625 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770642+0000", 2024-04-26T04:32:46.625 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.625 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.625 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.625 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.625 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770642+0000", 2024-04-26T04:32:46.625 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.625 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.625 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.625 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.625 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770644+0000", 2024-04-26T04:32:46.625 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.124e-06 2024-04-26T04:32:46.625 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.626 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.626 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.626 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770906+0000", 2024-04-26T04:32:46.626 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026233199999999999 2024-04-26T04:32:46.626 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.626 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.626 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.626 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770930+0000", 2024-04-26T04:32:46.626 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4111999999999999e-05 2024-04-26T04:32:46.626 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.626 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.626 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.626 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.770944+0000", 2024-04-26T04:32:46.627 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3566e-05 2024-04-26T04:32:46.627 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.627 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.627 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.627 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071117+0000", 2024-04-26T04:32:46.627 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30017346 2024-04-26T04:32:46.627 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.627 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.627 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.627 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071145+0000", 2024-04-26T04:32:46.627 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7801e-05 2024-04-26T04:32:46.627 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.627 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.628 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.628 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.628 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.628 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:247 3.8s0 3:1a4ccad9:::smithi17829308-40:head [append 0~573440] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:46.628 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.788189+0000", 2024-04-26T04:32:46.628 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.798981820000002, 2024-04-26T04:32:46.628 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.58923427699999997, 2024-04-26T04:32:46.628 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.628 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.628 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:46.628 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:46.628 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:46.628 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:46.629 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 247 2024-04-26T04:32:46.629 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.629 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.629 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.629 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.629 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.788189+0000", 2024-04-26T04:32:46.629 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.629 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.629 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.629 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.629 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.788189+0000", 2024-04-26T04:32:46.629 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.629 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.630 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.630 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.630 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.788190+0000", 2024-04-26T04:32:46.630 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0270000000000001e-06 2024-04-26T04:32:46.630 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.630 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.630 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.630 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.789159+0000", 2024-04-26T04:32:46.630 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00096895399999999995 2024-04-26T04:32:46.630 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.630 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.630 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.631 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.789190+0000", 2024-04-26T04:32:46.631 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0839000000000001e-05 2024-04-26T04:32:46.631 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.631 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.631 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.631 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.789205+0000", 2024-04-26T04:32:46.631 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5139e-05 2024-04-26T04:32:46.631 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.631 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.631 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.631 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072824+0000", 2024-04-26T04:32:46.631 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.283618659 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072888+0000", 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.4080999999999999e-05 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.073554+0000", 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00066656699999999996 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.632 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.077266+0000", 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0037117320000000001 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.377359+0000", 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30009325399999998 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.377423+0000", 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.4024999999999998e-05 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.633 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.634 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.634 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.634 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:248 3.8s0 3:1a4ccad9:::smithi17829308-40:head [append 0~376832] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:46.634 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.793207+0000", 2024-04-26T04:32:46.634 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.793963804000001, 2024-04-26T04:32:46.634 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.68466394600000002, 2024-04-26T04:32:46.634 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.634 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.634 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:46.634 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:46.634 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:46.634 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:46.634 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 248 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793207+0000", 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793207+0000", 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.635 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793209+0000", 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7269999999999999e-06 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793709+0000", 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000500247 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793730+0000", 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0931999999999999e-05 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.636 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.637 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.637 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793741+0000", 2024-04-26T04:32:46.637 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1032999999999999e-05 2024-04-26T04:32:46.637 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.637 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.637 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.637 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.074358+0000", 2024-04-26T04:32:46.637 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.28061672700000001 2024-04-26T04:32:46.637 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.637 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.637 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:46.637 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.074434+0000", 2024-04-26T04:32:46.637 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.6691000000000001e-05 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.075231+0000", 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00079667900000000001 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.077267+0000", 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0020362380000000001 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:46.638 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.477751+0000", 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.400483435 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.477871+0000", 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012023699999999999 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:249 3.8s0 3:1a4ccad9:::smithi17829308-40:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.793759+0000", 2024-04-26T04:32:46.639 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.793411765999998, 2024-04-26T04:32:46.640 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.68433669600000002, 2024-04-26T04:32:46.640 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.640 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.640 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:46.640 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:46.640 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:46.640 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:46.640 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 249 2024-04-26T04:32:46.640 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.640 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.640 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.640 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.640 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793759+0000", 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793759+0000", 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793760+0000", 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.043e-06 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.641 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.642 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793765+0000", 2024-04-26T04:32:46.642 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.51e-06 2024-04-26T04:32:46.642 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.642 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.642 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.642 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793775+0000", 2024-04-26T04:32:46.642 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0586e-05 2024-04-26T04:32:46.642 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.642 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.642 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.642 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893886+0000", 2024-04-26T04:32:46.642 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100111197 2024-04-26T04:32:46.642 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.075921+0000", 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.18203503800000001 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.076055+0000", 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013402999999999999 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.076679+0000", 2024-04-26T04:32:46.643 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00062310000000000002 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.178214+0000", 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101535002 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478030+0000", 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.29981632200000002 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.644 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478096+0000", 2024-04-26T04:32:46.645 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.5868e-05 2024-04-26T04:32:46.645 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.645 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.645 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.645 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.645 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.645 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=38, attrs_read=0))", 2024-04-26T04:32:46.645 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:46.645 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.716844626, 2024-04-26T04:32:46.645 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20188995300000001, 2024-04-26T04:32:46.645 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.645 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.645 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.646 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.646 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.646 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.646 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:46.646 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.646 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.646 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.646 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.646 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:46.646 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.646 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.646 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.647 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.647 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:46.647 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.647 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.647 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.647 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.647 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870326+0000", 2024-04-26T04:32:46.647 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.647 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.647 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.647 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.647 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870347+0000", 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0727999999999999e-05 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.970445+0000", 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100097987 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072194+0000", 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101748964 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.648 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072216+0000", 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2274000000000001e-05 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=38, attrs_read=0))", 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870408+0000", 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.716763289999999, 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20239011900000001, 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.649 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870408+0000", 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870408+0000", 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870409+0000", 2024-04-26T04:32:46.650 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3090000000000001e-06 2024-04-26T04:32:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870541+0000", 2024-04-26T04:32:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000131815 2024-04-26T04:32:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870551+0000", 2024-04-26T04:32:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0553e-05 2024-04-26T04:32:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.651 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.652 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.970635+0000", 2024-04-26T04:32:46.652 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10008423800000001 2024-04-26T04:32:46.652 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.652 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.652 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.652 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072252+0000", 2024-04-26T04:32:46.652 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10161698800000001 2024-04-26T04:32:46.652 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.652 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.652 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.652 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072798+0000", 2024-04-26T04:32:46.652 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00054521599999999995 2024-04-26T04:32:46.653 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.653 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.653 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.653 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.653 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.653 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-26T04:32:46.653 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870458+0000", 2024-04-26T04:32:46.653 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.716712711, 2024-04-26T04:32:46.653 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201580751, 2024-04-26T04:32:46.653 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.653 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.653 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.654 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.654 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.654 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.654 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870458+0000", 2024-04-26T04:32:46.654 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.654 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.654 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.654 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.654 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870458+0000", 2024-04-26T04:32:46.654 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.655 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.655 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.655 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.655 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870459+0000", 2024-04-26T04:32:46.655 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.375e-06 2024-04-26T04:32:46.655 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.655 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.655 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.655 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870589+0000", 2024-04-26T04:32:46.655 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012998600000000001 2024-04-26T04:32:46.655 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.655 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.655 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.656 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870610+0000", 2024-04-26T04:32:46.656 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0163999999999999e-05 2024-04-26T04:32:46.656 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.656 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.656 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.656 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870625+0000", 2024-04-26T04:32:46.656 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5339e-05 2024-04-26T04:32:46.656 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.656 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.656 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.656 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.071721+0000", 2024-04-26T04:32:46.656 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20109591299999999 2024-04-26T04:32:46.656 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.657 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.657 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.657 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072039+0000", 2024-04-26T04:32:46.657 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000317974 2024-04-26T04:32:46.657 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.657 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.657 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.657 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.657 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.657 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpReadReply(3.8s0 36/30 ECSubReadReply(tid=38, attrs_read=0))", 2024-04-26T04:32:46.657 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.870645+0000", 2024-04-26T04:32:46.657 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.716525611000002, 2024-04-26T04:32:46.658 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20152561399999999, 2024-04-26T04:32:46.658 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.658 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.658 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.658 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.658 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.658 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.658 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870645+0000", 2024-04-26T04:32:46.658 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.658 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.658 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.658 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.658 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870645+0000", 2024-04-26T04:32:46.659 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.659 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.659 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.659 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.659 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870646+0000", 2024-04-26T04:32:46.659 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.7800000000000001e-07 2024-04-26T04:32:46.659 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.659 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.659 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.659 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870838+0000", 2024-04-26T04:32:46.659 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00019167500000000001 2024-04-26T04:32:46.659 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870855+0000", 2024-04-26T04:32:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7730000000000001e-05 2024-04-26T04:32:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.870868+0000", 2024-04-26T04:32:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2308e-05 2024-04-26T04:32:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.660 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072101+0000", 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201233511 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072171+0000", 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.9711999999999999e-05 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:250 3.8s0 3:1a4ccad9:::smithi17829308-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.893915+0000", 2024-04-26T04:32:46.661 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.693256311999999, 2024-04-26T04:32:46.662 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.66902760400000005, 2024-04-26T04:32:46.662 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.662 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.662 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:32:46.662 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:32:46.662 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:32:46.662 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:46.662 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 250 2024-04-26T04:32:46.662 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.662 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.662 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.662 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.662 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893915+0000", 2024-04-26T04:32:46.663 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.663 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.663 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.663 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.663 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893915+0000", 2024-04-26T04:32:46.663 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.663 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.663 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.663 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.663 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893917+0000", 2024-04-26T04:32:46.663 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3029999999999998e-06 2024-04-26T04:32:46.663 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893925+0000", 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4370000000000007e-06 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893965+0000", 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9941000000000003e-05 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893976+0000", 2024-04-26T04:32:46.664 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1049999999999999e-05 2024-04-26T04:32:46.665 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.665 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.665 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.665 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.177679+0000", 2024-04-26T04:32:46.665 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.28370258100000001 2024-04-26T04:32:46.665 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.665 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.665 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:46.665 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.177965+0000", 2024-04-26T04:32:46.665 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000286309 2024-04-26T04:32:46.665 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.665 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.665 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.666 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478480+0000", 2024-04-26T04:32:46.666 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30051439600000002 2024-04-26T04:32:46.666 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.666 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.666 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:32:46.666 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478666+0000", 2024-04-26T04:32:46.666 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018635599999999999 2024-04-26T04:32:46.666 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.666 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.666 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.666 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.562942+0000", 2024-04-26T04:32:46.666 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.084276230999999993 2024-04-26T04:32:46.666 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.667 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.667 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.667 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.667 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.667 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s3 36/30 ECSubWrite(tid=78, reqid=client.4670.0:261, at_version=36'29, trim_to=0'0, roll_forward_to=36'22))", 2024-04-26T04:32:46.667 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.559935+0000", 2024-04-26T04:32:46.667 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.027235659999999, 2024-04-26T04:32:46.667 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201239479, 2024-04-26T04:32:46.667 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.667 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.667 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.667 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.667 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.559935+0000", 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.559935+0000", 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.559936+0000", 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.9800000000000002e-07 2024-04-26T04:32:46.668 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.669 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.669 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.669 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.559962+0000", 2024-04-26T04:32:46.669 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5842000000000001e-05 2024-04-26T04:32:46.669 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.669 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.669 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.669 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.559975+0000", 2024-04-26T04:32:46.669 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3278999999999999e-05 2024-04-26T04:32:46.669 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.669 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.669 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.670 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.559983+0000", 2024-04-26T04:32:46.670 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.841e-06 2024-04-26T04:32:46.670 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.670 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.670 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.670 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.660143+0000", 2024-04-26T04:32:46.670 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100160113 2024-04-26T04:32:46.670 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.670 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.670 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.670 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.660156+0000", 2024-04-26T04:32:46.671 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2386e-05 2024-04-26T04:32:46.671 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.671 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.671 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.671 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.761106+0000", 2024-04-26T04:32:46.671 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100950842 2024-04-26T04:32:46.671 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.671 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.671 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.671 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.761175+0000", 2024-04-26T04:32:46.671 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.8177999999999997e-05 2024-04-26T04:32:46.671 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.672 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.672 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.672 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.672 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.672 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s3 36/30 ECSubWrite(tid=79, reqid=client.4670.0:251, at_version=36'30, trim_to=0'0, roll_forward_to=36'22))", 2024-04-26T04:32:46.672 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.559992+0000", 2024-04-26T04:32:46.672 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.027178421999999, 2024-04-26T04:32:46.672 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20175707100000001, 2024-04-26T04:32:46.672 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.672 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.672 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.672 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.559992+0000", 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.559992+0000", 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.559993+0000", 2024-04-26T04:32:46.673 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.7e-07 2024-04-26T04:32:46.674 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.674 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.674 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.674 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.560105+0000", 2024-04-26T04:32:46.674 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000112428 2024-04-26T04:32:46.674 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.674 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.674 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.674 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.560115+0000", 2024-04-26T04:32:46.674 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3649999999999993e-06 2024-04-26T04:32:46.674 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.674 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.674 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.675 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.560120+0000", 2024-04-26T04:32:46.675 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.4820000000000002e-06 2024-04-26T04:32:46.675 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.675 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.675 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.675 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.760599+0000", 2024-04-26T04:32:46.675 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20047927500000001 2024-04-26T04:32:46.675 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.675 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.675 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.675 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.760608+0000", 2024-04-26T04:32:46.675 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.541e-06 2024-04-26T04:32:46.675 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.761678+0000", 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0010697700000000001 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.761749+0000", 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.1840000000000003e-05 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.676 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.7s5 46/30 ECSubWrite(tid=143, reqid=osd.4.0:142, at_version=0'0, trim_to=0'0, roll_forward_to=46'21))", 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:40.653577+0000", 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.9335933839999999, 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201561083, 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.653577+0000", 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.677 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.678 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.678 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.653577+0000", 2024-04-26T04:32:46.678 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.678 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.678 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.678 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.678 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.653579+0000", 2024-04-26T04:32:46.678 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9800000000000001e-06 2024-04-26T04:32:46.678 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.678 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.678 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.678 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.653601+0000", 2024-04-26T04:32:46.678 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1532e-05 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.653626+0000", 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4929000000000001e-05 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.753772+0000", 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100146361 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.679 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.854073+0000", 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100300554 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.854098+0000", 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5432e-05 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.855084+0000", 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00098568399999999991 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.680 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.681 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.855139+0000", 2024-04-26T04:32:46.681 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.4611000000000003e-05 2024-04-26T04:32:46.681 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.681 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.681 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.681 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.681 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.681 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 50/30 ECSubWriteReply(tid=65, last_complete=50'53, committed=1, applied=1))", 2024-04-26T04:32:46.681 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:44.685273+0000", 2024-04-26T04:32:46.681 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.901898101, 2024-04-26T04:32:46.681 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.202464476, 2024-04-26T04:32:46.681 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:46.681 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:46.682 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:46.682 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:46.682 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.682 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:46.682 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.685273+0000", 2024-04-26T04:32:46.682 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.682 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.682 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.682 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:46.682 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.685273+0000", 2024-04-26T04:32:46.682 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:46.682 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.685275+0000", 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8470000000000001e-06 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.685284+0000", 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7289999999999995e-06 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.685314+0000", 2024-04-26T04:32:46.683 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0125e-05 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.785456+0000", 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10014200199999999 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.887701+0000", 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102244164 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:46.684 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.887737+0000", 2024-04-26T04:32:46.685 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.6609000000000001e-05 2024-04-26T04:32:46.685 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.685 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.685 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.685 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:46.685 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:46.685 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:46.687 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T04:32:46.687 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:46.682+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:46.692 INFO:tasks.rados.rados.0.smithi178.stdout:172: finishing write tid 1 to smithi17829308-6 2024-04-26T04:32:46.693 INFO:tasks.rados.rados.0.smithi178.stdout:172: finishing write tid 2 to smithi17829308-6 2024-04-26T04:32:46.735 INFO:tasks.rados.rados.0.smithi178.stdout:172: finishing write tid 3 to smithi17829308-6 2024-04-26T04:32:46.740 INFO:tasks.rados.rados.0.smithi178.stdout:173: finishing write tid 1 to smithi17829308-10 2024-04-26T04:32:46.741 INFO:tasks.rados.rados.0.smithi178.stdout:173: finishing write tid 2 to smithi17829308-10 2024-04-26T04:32:46.741 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 50 v 18 (ObjNum 71 snap 3 seq_num 71) dirty exists 2024-04-26T04:32:46.742 INFO:tasks.rados.rados.0.smithi178.stdout:173: finishing write tid 3 to smithi17829308-10 2024-04-26T04:32:46.743 INFO:tasks.rados.rados.0.smithi178.stdout:173: finishing write tid 4 to smithi17829308-10 2024-04-26T04:32:46.746 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:46.741+0000 7fc591d18640 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 9 after trimming was started 2024-04-26T04:32:46.753 INFO:tasks.rados.rados.0.smithi178.stdout:175: expect (ObjNum 78 snap 8 seq_num 78) 2024-04-26T04:32:46.787 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:46.784+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:46.851 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:46.851 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:46.852 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 0 2024-04-26T04:32:46.852 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:46.865 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T04:32:46.888 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:46.886+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:46.970 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T04:32:46.989 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:46.986+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:46.990 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:46.990 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:32:46.991 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:32:46.991 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:32:46.991 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:46.998 INFO:tasks.rados.rados.0.smithi178.stdout:176: finishing write tid 1 to smithi17829308-2 2024-04-26T04:32:46.998 INFO:tasks.rados.rados.0.smithi178.stdout:176: finishing write tid 2 to smithi17829308-2 2024-04-26T04:32:46.998 INFO:tasks.rados.rados.0.smithi178.stdout:176: finishing write tid 3 to smithi17829308-2 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:176: finishing write tid 4 to smithi17829308-2 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:172: finishing write tid 4 to smithi17829308-6 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:173: finishing write tid 5 to smithi17829308-10 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:173: oid 10 updating version 0 to 46 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:173: oid 10 version 46 is already newer than 45 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:173: oid 10 version 46 is already newer than 44 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:173: oid 10 version 46 is already newer than 43 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 10 v 46 (ObjNum 83 snap 9 seq_num 83) dirty exists 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:173: left oid 10 (ObjNum 83 snap 9 seq_num 83) 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:176: finishing write tid 5 to smithi17829308-2 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:176: oid 2 updating version 0 to 52 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:176: oid 2 version 52 is already newer than 51 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:176: oid 2 version 52 is already newer than 50 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:176: oid 2 updating version 52 to 53 2024-04-26T04:32:46.999 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 2 v 53 (ObjNum 84 snap 9 seq_num 84) dirty exists 2024-04-26T04:32:47.000 INFO:tasks.rados.rados.0.smithi178.stdout:176: left oid 2 (ObjNum 84 snap 9 seq_num 84) 2024-04-26T04:32:47.005 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T04:32:47.090 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:47.086+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:47.131 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:32:47.131 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:32:47.132 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:32:47.132 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:32:47.132 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.132 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 32/30 ECSubWrite(tid=1, reqid=client.4670.0:11, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:47.132 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.209171+0000", 2024-04-26T04:32:47.132 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 21.920644854999999, 2024-04-26T04:32:47.132 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19807992999999999, 2024-04-26T04:32:47.132 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.132 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.132 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.132 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.132 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.132 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209171+0000", 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209171+0000", 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209173+0000", 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1440000000000001e-06 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.133 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.134 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.134 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209220+0000", 2024-04-26T04:32:47.134 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.7030000000000002e-05 2024-04-26T04:32:47.134 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.134 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.134 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.134 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209250+0000", 2024-04-26T04:32:47.134 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0281999999999999e-05 2024-04-26T04:32:47.134 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.134 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.134 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209281+0000", 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0895000000000002e-05 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.405953+0000", 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19667172599999999 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.405979+0000", 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6452e-05 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.135 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.407083+0000", 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0011043139999999999 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.407250+0000", 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016708700000000001 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.136 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s2 32/30 ECSubWrite(tid=1, reqid=client.4670.0:5, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:47.137 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.312729+0000", 2024-04-26T04:32:47.137 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 21.817086554999999, 2024-04-26T04:32:47.137 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20126603100000001, 2024-04-26T04:32:47.137 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.137 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.137 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.137 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.137 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.137 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.137 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312729+0000", 2024-04-26T04:32:47.137 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.137 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.137 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312729+0000", 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312731+0000", 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9130000000000001e-06 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312766+0000", 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4768000000000003e-05 2024-04-26T04:32:47.138 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312797+0000", 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1201000000000002e-05 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312814+0000", 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7073000000000001e-05 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.413078+0000", 2024-04-26T04:32:47.139 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10026389400000001 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.413092+0000", 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4188999999999999e-05 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.513794+0000", 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100702298 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.513995+0000", 2024-04-26T04:32:47.140 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000200695 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es1 32/30 ECSubWrite(tid=158, reqid=client.4670.0:32, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.316326+0000", 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 21.813489642, 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102271657, 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.141 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.142 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316326+0000", 2024-04-26T04:32:47.142 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.142 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.142 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.142 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.142 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316326+0000", 2024-04-26T04:32:47.142 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.142 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.142 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.142 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.142 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316327+0000", 2024-04-26T04:32:47.142 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.454e-06 2024-04-26T04:32:47.142 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316361+0000", 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4060999999999998e-05 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316387+0000", 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5395e-05 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316421+0000", 2024-04-26T04:32:47.143 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4205000000000002e-05 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.416649+0000", 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100228373 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.416673+0000", 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3872000000000001e-05 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.418462+0000", 2024-04-26T04:32:47.144 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0017885710000000001 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.418597+0000", 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013572599999999999 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 33/30 ECSubWrite(tid=2, reqid=client.4670.0:28, at_version=33'3, trim_to=0'0, roll_forward_to=32'1))", 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.508663+0000", 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 21.621151921999999, 2024-04-26T04:32:47.145 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102330728, 2024-04-26T04:32:47.146 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.146 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.146 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.146 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.146 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.146 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.146 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508663+0000", 2024-04-26T04:32:47.146 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.146 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.146 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.146 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.146 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508663+0000", 2024-04-26T04:32:47.146 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508664+0000", 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.0999999999999999e-07 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508841+0000", 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017671599999999999 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508866+0000", 2024-04-26T04:32:47.147 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4879e-05 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.508882+0000", 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6458999999999999e-05 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509022+0000", 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014024099999999999 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.148 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.509036+0000", 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.419e-05 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.610954+0000", 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10191800500000001 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.610994+0000", 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9728000000000003e-05 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.149 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.150 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.150 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.7s2 35/30 ECSubRead(tid=58, to_read={3:e1054bf1:::smithi17829308-37:head=0,151552,0}, subchunks={3:e1054bf1:::smithi17829308-37:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:47.150 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.954839+0000", 2024-04-26T04:32:47.150 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 18.174976815000001, 2024-04-26T04:32:47.150 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20146535800000001, 2024-04-26T04:32:47.150 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.150 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.150 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.150 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.150 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.150 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.150 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954839+0000", 2024-04-26T04:32:47.150 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954839+0000", 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954840+0000", 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.826e-06 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954857+0000", 2024-04-26T04:32:47.151 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6333000000000001e-05 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954890+0000", 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.3594999999999999e-05 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954912+0000", 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1325e-05 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.152 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.155338+0000", 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20042677 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.156304+0000", 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00096550900000000003 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs6 35/30 ECSubWrite(tid=59, reqid=client.4670.0:176, at_version=0'0, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.055093+0000", 2024-04-26T04:32:47.153 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 18.074722827999999, 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201719765, 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055093+0000", 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055093+0000", 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.154 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055095+0000", 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.227e-06 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055119+0000", 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4329e-05 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055151+0000", 2024-04-26T04:32:47.155 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1507999999999997e-05 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.155266+0000", 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100114996 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.255601+0000", 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100334886 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.255617+0000", 2024-04-26T04:32:47.156 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6446000000000001e-05 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.256718+0000", 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001101524 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.256812+0000", 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3849000000000002e-05 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.157 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es1 35/30 ECSubWrite(tid=190, reqid=client.4670.0:235, at_version=0'0, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.421980+0000", 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 17.707835663000001, 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20113165399999999, 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421980+0000", 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.158 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421980+0000", 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421982+0000", 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2639999999999998e-06 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.422038+0000", 2024-04-26T04:32:47.159 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.6273000000000001e-05 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.422071+0000", 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2437999999999997e-05 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.522157+0000", 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100086417 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.160 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.622323+0000", 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100166121 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.622332+0000", 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.1309999999999995e-06 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.623023+0000", 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000690491 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.161 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.623111+0000", 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.8518999999999997e-05 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s2 36/30 ECSubWrite(tid=64, reqid=client.4670.0:237, at_version=36'20, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.943082+0000", 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 17.186732932999998, 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102547737, 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.162 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.943082+0000", 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.943082+0000", 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.943085+0000", 2024-04-26T04:32:47.163 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1330000000000002e-06 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.943380+0000", 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00029546100000000001 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.943426+0000", 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.6224000000000003e-05 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.164 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.043544+0000", 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10011753399999999 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.043755+0000", 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021160100000000001 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.043780+0000", 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4454999999999999e-05 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.165 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.045600+0000", 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001819878 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.045630+0000", 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0451e-05 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s2 41/30 ECSubWrite(tid=115, reqid=client.4670.0:355, at_version=41'36, trim_to=0'0, roll_forward_to=40'34))", 2024-04-26T04:32:47.166 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.444369+0000", 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 11.685446749, 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20274193700000001, 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444369+0000", 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.167 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444369+0000", 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444371+0000", 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9099999999999999e-06 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444630+0000", 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00025897499999999999 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.168 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444655+0000", 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5819000000000001e-05 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444670+0000", 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4779e-05 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.644608+0000", 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19993833 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.169 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.644623+0000", 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4806e-05 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.647032+0000", 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0024082880000000002 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.647111+0000", 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.9029999999999994e-05 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.170 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s2 41/30 ECSubWrite(tid=116, reqid=client.4670.0:356, at_version=41'37, trim_to=0'0, roll_forward_to=40'34))", 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.447328+0000", 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 11.682487736000001, 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.200159164, 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.447328+0000", 2024-04-26T04:32:47.171 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.172 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.172 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.172 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.172 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.447328+0000", 2024-04-26T04:32:47.172 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.173 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.173 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.173 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.173 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.447329+0000", 2024-04-26T04:32:47.173 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6500000000000001e-06 2024-04-26T04:32:47.173 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.173 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.173 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.173 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.447476+0000", 2024-04-26T04:32:47.173 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014682700000000001 2024-04-26T04:32:47.173 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.173 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.173 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.174 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.447506+0000", 2024-04-26T04:32:47.174 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9694999999999999e-05 2024-04-26T04:32:47.174 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.174 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.174 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.174 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.447521+0000", 2024-04-26T04:32:47.174 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5383000000000002e-05 2024-04-26T04:32:47.174 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.174 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.174 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.174 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.645443+0000", 2024-04-26T04:32:47.174 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19792164700000001 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.645469+0000", 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6602999999999999e-05 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.647413+0000", 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001943492 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.175 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.647487+0000", 2024-04-26T04:32:47.176 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3867000000000003e-05 2024-04-26T04:32:47.176 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.176 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.176 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.176 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.176 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.176 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.7s2 45/30 ECSubWrite(tid=137, reqid=client.4670.0:371, at_version=45'18, trim_to=0'0, roll_forward_to=44'15))", 2024-04-26T04:32:47.176 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.608809+0000", 2024-04-26T04:32:47.176 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 7.5210064980000002, 2024-04-26T04:32:47.176 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20229014100000001, 2024-04-26T04:32:47.176 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.176 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.176 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.177 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.177 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.177 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.177 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608809+0000", 2024-04-26T04:32:47.177 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.177 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.177 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.177 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.177 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608809+0000", 2024-04-26T04:32:47.177 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.178 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.178 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.178 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.178 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608811+0000", 2024-04-26T04:32:47.178 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2730000000000001e-06 2024-04-26T04:32:47.178 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.178 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.178 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.178 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609164+0000", 2024-04-26T04:32:47.178 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00035305999999999998 2024-04-26T04:32:47.178 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.178 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.178 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.179 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609195+0000", 2024-04-26T04:32:47.179 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0818000000000002e-05 2024-04-26T04:32:47.179 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.179 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.179 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.179 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609214+0000", 2024-04-26T04:32:47.179 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8819000000000001e-05 2024-04-26T04:32:47.179 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.179 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.179 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.179 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.709463+0000", 2024-04-26T04:32:47.179 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10024947300000001 2024-04-26T04:32:47.180 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.180 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.180 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.180 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.709490+0000", 2024-04-26T04:32:47.180 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6312000000000002e-05 2024-04-26T04:32:47.180 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.180 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.180 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.180 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.811034+0000", 2024-04-26T04:32:47.180 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10154439899999999 2024-04-26T04:32:47.180 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.811099+0000", 2024-04-26T04:32:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.4987000000000006e-05 2024-04-26T04:32:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.7s2 45/30 ECSubWrite(tid=138, reqid=client.4670.0:372, at_version=45'19, trim_to=0'0, roll_forward_to=44'15))", 2024-04-26T04:32:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.609251+0000", 2024-04-26T04:32:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 7.5205640030000005, 2024-04-26T04:32:47.181 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20191711300000001, 2024-04-26T04:32:47.182 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.182 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.182 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.182 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.182 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.182 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.182 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609251+0000", 2024-04-26T04:32:47.182 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.182 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.182 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.182 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.182 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609251+0000", 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609252+0000", 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2999999999999999e-07 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609276+0000", 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3703e-05 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.183 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609292+0000", 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6535999999999999e-05 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609303+0000", 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0539e-05 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.710393+0000", 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101089625 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.184 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.710408+0000", 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5079000000000001e-05 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.811107+0000", 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10069903099999999 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.811169+0000", 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.177e-05 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.185 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.186 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.186 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.186 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 45/30 ECSubWrite(tid=35, reqid=client.4670.0:377, at_version=0'0, trim_to=0'0, roll_forward_to=45'23))", 2024-04-26T04:32:47.186 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:40.134471+0000", 2024-04-26T04:32:47.186 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.9953442040000002, 2024-04-26T04:32:47.186 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20268414700000001, 2024-04-26T04:32:47.186 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.186 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.186 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.186 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.186 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.186 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.186 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134471+0000", 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134471+0000", 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134473+0000", 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2000000000000001e-06 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.187 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.188 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134501+0000", 2024-04-26T04:32:47.188 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7441999999999999e-05 2024-04-26T04:32:47.188 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.188 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.188 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.188 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134540+0000", 2024-04-26T04:32:47.188 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9181999999999999e-05 2024-04-26T04:32:47.188 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.188 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.188 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.188 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234639+0000", 2024-04-26T04:32:47.188 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100099332 2024-04-26T04:32:47.188 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.189 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.189 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.189 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234798+0000", 2024-04-26T04:32:47.189 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015831999999999999 2024-04-26T04:32:47.189 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.189 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.189 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.189 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234822+0000", 2024-04-26T04:32:47.189 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4766e-05 2024-04-26T04:32:47.189 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.189 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.189 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.190 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.337076+0000", 2024-04-26T04:32:47.190 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102253652 2024-04-26T04:32:47.190 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.190 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.190 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.190 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.337155+0000", 2024-04-26T04:32:47.190 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.9252999999999995e-05 2024-04-26T04:32:47.190 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.190 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.190 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.190 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.190 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.190 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 45/30 ECSubWrite(tid=36, reqid=client.4670.0:379, at_version=45'25, trim_to=0'0, roll_forward_to=45'23))", 2024-04-26T04:32:47.191 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:40.234694+0000", 2024-04-26T04:32:47.191 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.895121273, 2024-04-26T04:32:47.191 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.2029495, 2024-04-26T04:32:47.191 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.191 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.191 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.191 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.191 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.191 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.191 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234694+0000", 2024-04-26T04:32:47.191 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.191 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234694+0000", 2024-04-26T04:32:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234695+0000", 2024-04-26T04:32:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.356e-06 2024-04-26T04:32:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.192 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235050+0000", 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00035439899999999998 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235076+0000", 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5619e-05 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235096+0000", 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0046000000000001e-05 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.193 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.194 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.335458+0000", 2024-04-26T04:32:47.194 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10036199799999999 2024-04-26T04:32:47.194 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.194 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.194 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.194 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.335472+0000", 2024-04-26T04:32:47.194 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.472e-05 2024-04-26T04:32:47.194 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.194 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.194 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.194 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.437566+0000", 2024-04-26T04:32:47.194 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102093589 2024-04-26T04:32:47.194 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.195 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.195 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.195 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.437644+0000", 2024-04-26T04:32:47.195 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7773000000000002e-05 2024-04-26T04:32:47.195 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.195 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.195 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.195 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.195 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.195 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.4s1 45/30 ECSubRead(tid=37, to_read={3:26eb43ca:::smithi17829308-16:head=0,4096,0}, subchunks={3:26eb43ca:::smithi17829308-16:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:47.195 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:40.235130+0000", 2024-04-26T04:32:47.195 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.8946856509999996, 2024-04-26T04:32:47.196 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20238550699999999, 2024-04-26T04:32:47.196 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.196 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.196 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.196 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.196 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.196 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.196 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235130+0000", 2024-04-26T04:32:47.196 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.196 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.197 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.197 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.197 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235130+0000", 2024-04-26T04:32:47.197 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.197 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.197 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.197 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.197 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235131+0000", 2024-04-26T04:32:47.197 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1739999999999999e-06 2024-04-26T04:32:47.197 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.198 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.198 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.198 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235146+0000", 2024-04-26T04:32:47.198 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5175e-05 2024-04-26T04:32:47.198 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.198 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.198 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.198 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235172+0000", 2024-04-26T04:32:47.198 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5406e-05 2024-04-26T04:32:47.198 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.198 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.198 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.198 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235195+0000", 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3266000000000001e-05 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.437356+0000", 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20216114600000001 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.437515+0000", 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015934 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.199 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 47/30 ECSubWrite(tid=42, reqid=osd.8.0:42, at_version=47'31, trim_to=0'0, roll_forward_to=46'29))", 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:41.248219+0000", 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.8815965309999996, 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20129562100000001, 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248219+0000", 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.200 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248219+0000", 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248220+0000", 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5400000000000001e-06 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248271+0000", 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.0377999999999998e-05 2024-04-26T04:32:47.201 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248305+0000", 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4598999999999999e-05 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348396+0000", 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10009045699999999 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348535+0000", 2024-04-26T04:32:47.202 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000138975 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348560+0000", 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5333e-05 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.449434+0000", 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100873729 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.449514+0000", 2024-04-26T04:32:47.203 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0610000000000002e-05 2024-04-26T04:32:47.204 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.204 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.204 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.204 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.204 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.204 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 47/30 ECSubWrite(tid=43, reqid=osd.8.0:43, at_version=47'33, trim_to=0'0, roll_forward_to=46'29))", 2024-04-26T04:32:47.204 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:41.348459+0000", 2024-04-26T04:32:47.204 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.7813564790000003, 2024-04-26T04:32:47.204 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10489925899999999, 2024-04-26T04:32:47.204 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.204 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.204 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.204 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348459+0000", 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348459+0000", 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348461+0000", 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7969999999999999e-06 2024-04-26T04:32:47.205 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.206 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.206 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.206 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348514+0000", 2024-04-26T04:32:47.206 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.3090999999999997e-05 2024-04-26T04:32:47.206 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.206 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.206 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.206 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348539+0000", 2024-04-26T04:32:47.206 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5211999999999998e-05 2024-04-26T04:32:47.206 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.206 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.206 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.206 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348559+0000", 2024-04-26T04:32:47.207 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9896000000000001e-05 2024-04-26T04:32:47.207 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.207 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.207 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.207 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.448802+0000", 2024-04-26T04:32:47.207 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100242807 2024-04-26T04:32:47.207 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.207 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.207 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.207 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.448817+0000", 2024-04-26T04:32:47.207 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5673000000000001e-05 2024-04-26T04:32:47.207 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.208 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.208 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.208 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.453296+0000", 2024-04-26T04:32:47.208 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0044787489999999998 2024-04-26T04:32:47.208 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.208 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.208 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.208 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.453358+0000", 2024-04-26T04:32:47.208 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.2033999999999996e-05 2024-04-26T04:32:47.208 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.208 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.208 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.208 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.209 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.209 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.as3 47/30 ECSubWrite(tid=37, reqid=osd.3.0:37, at_version=47'29, trim_to=0'0, roll_forward_to=46'27))", 2024-04-26T04:32:47.209 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:41.348863+0000", 2024-04-26T04:32:47.209 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.780952525, 2024-04-26T04:32:47.209 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10448713900000001, 2024-04-26T04:32:47.209 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.209 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.209 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.209 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.209 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.209 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.209 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348863+0000", 2024-04-26T04:32:47.209 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348863+0000", 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348863+0000", 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.3399999999999999e-07 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348913+0000", 2024-04-26T04:32:47.210 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.0037999999999998e-05 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348935+0000", 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1582000000000001e-05 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.449082+0000", 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100147108 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.211 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.449237+0000", 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015464999999999999 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.449252+0000", 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5206e-05 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.453286+0000", 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0040337799999999998 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.212 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.453350+0000", 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.4240999999999997e-05 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 52/30 ECSubWrite(tid=54, reqid=client.4670.0:435, at_version=52'43, trim_to=0'0, roll_forward_to=50'41))", 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:46.228660+0000", 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.90115571699999997, 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.108257319, 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.213 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.228660+0000", 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.228660+0000", 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.228660+0000", 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.9900000000000001e-07 2024-04-26T04:32:47.214 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.229088+0000", 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00042799800000000002 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.229112+0000", 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3779000000000001e-05 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.329218+0000", 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100106193 2024-04-26T04:32:47.215 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.332933+0000", 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0037152370000000001 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.332949+0000", 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5189e-05 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.336872+0000", 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0039229590000000002 2024-04-26T04:32:47.216 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.217 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.217 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.217 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.336917+0000", 2024-04-26T04:32:47.217 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.5464999999999999e-05 2024-04-26T04:32:47.217 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.217 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.217 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.217 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.217 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.217 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 52/30 ECSubWrite(tid=57, reqid=client.4670.0:438, at_version=52'46, trim_to=0'0, roll_forward_to=50'41))", 2024-04-26T04:32:47.218 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:46.329453+0000", 2024-04-26T04:32:47.218 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.80036258000000005, 2024-04-26T04:32:47.218 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.107403019, 2024-04-26T04:32:47.218 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:32:47.218 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:32:47.218 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:32:47.218 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:32:47.218 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.218 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:32:47.218 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.329453+0000", 2024-04-26T04:32:47.218 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.218 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.218 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.329453+0000", 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.329453+0000", 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.92e-07 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.329460+0000", 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0690000000000004e-06 2024-04-26T04:32:47.219 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.329464+0000", 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9689999999999996e-06 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.329469+0000", 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.198e-06 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.432953+0000", 2024-04-26T04:32:47.220 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10348344700000001 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.432983+0000", 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0182e-05 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.436766+0000", 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0037831850000000001 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.436856+0000", 2024-04-26T04:32:47.221 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.9777000000000005e-05 2024-04-26T04:32:47.222 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.222 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.222 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.222 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:32:47.222 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:32:47.222 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:32:47.223 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:47.224 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:47.224 INFO:tasks.rados.rados.0.smithi178.stdout:173: done (5 left) 2024-04-26T04:32:47.224 INFO:tasks.rados.rados.0.smithi178.stdout:174: done (4 left) 2024-04-26T04:32:47.224 INFO:tasks.rados.rados.0.smithi178.stdout:175: done (3 left) 2024-04-26T04:32:47.224 INFO:tasks.rados.rados.0.smithi178.stdout:176: done (2 left) 2024-04-26T04:32:47.225 INFO:tasks.rados.rados.0.smithi178.stdout:177: done (1 left) 2024-04-26T04:32:47.225 INFO:tasks.rados.rados.0.smithi178.stdout:178: snap_create 2024-04-26T04:32:47.225 INFO:tasks.rados.rados.0.smithi178.stdout:172: finishing write tid 5 to smithi17829308-6 2024-04-26T04:32:47.225 INFO:tasks.rados.rados.0.smithi178.stdout:172: oid 6 updating version 0 to 36 2024-04-26T04:32:47.225 INFO:tasks.rados.rados.0.smithi178.stdout:172: oid 6 updating version 36 to 37 2024-04-26T04:32:47.225 INFO:tasks.rados.rados.0.smithi178.stdout:172: oid 6 updating version 37 to 38 2024-04-26T04:32:47.225 INFO:tasks.rados.rados.0.smithi178.stdout:172: oid 6 version 38 is already newer than 35 2024-04-26T04:32:47.225 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 6 v 38 (ObjNum 82 snap 9 seq_num 82) dirty exists 2024-04-26T04:32:47.225 INFO:tasks.rados.rados.0.smithi178.stdout:172: left oid 6 (ObjNum 82 snap 9 seq_num 82) 2024-04-26T04:32:47.226 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:47.226 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:47.226 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T04:32:47.227 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:47.221+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-04-26T04:32:47.227 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.9 to 1 (10000) 2024-04-26T04:32:47.227 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:47.228 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:47.229 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-04-26T04:32:47.229 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:47.230 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:47.230 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:47.230 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:47.231 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:47.231 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:47.250 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:47.251 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:47.273 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:47.273 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:47.293 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:47.293 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:47.311 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:47.311 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:47.324 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:47.321+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:47.332 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:47.333 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:47.354 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:47.354 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:47.374 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:47.374 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:47.396 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:47.397 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:47.397 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:47.397 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:47.411 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T04:32:47.417 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:47.417 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:47.438 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:47.438 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:47.456 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:47.457+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:47.526 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:47.524+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:47.530 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:47.530 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:47.544 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T04:32:47.628 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:47.625+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:47.714 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:47.714 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:47.729 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T04:32:47.729 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:47.725+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:47.829 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:47.824+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:47.860 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:47.860 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:47.874 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T04:32:47.930 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:47.929+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:48.031 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:48.029+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:48.042 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:48.042 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:48.056 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T04:32:48.133 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:48.133+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:48.171 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.10 to 1 (10000) 2024-04-26T04:32:48.174 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:48.174 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:48.179 INFO:tasks.rados.rados.0.smithi178.stdout:172: done (1 left) 2024-04-26T04:32:48.179 INFO:tasks.rados.rados.0.smithi178.stdout:178: done (0 left) 2024-04-26T04:32:48.179 INFO:tasks.rados.rados.0.smithi178.stdout:179: read oid 45 snap -1 2024-04-26T04:32:48.179 INFO:tasks.rados.rados.0.smithi178.stdout:179: expect (ObjNum 79 snap 8 seq_num 79) 2024-04-26T04:32:48.179 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 31 current snap is 11 2024-04-26T04:32:48.179 INFO:tasks.rados.rados.0.smithi178.stdout:180: seq_num 85 ranges {4276224=671744,4947968=770048} 2024-04-26T04:32:48.190 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T04:32:48.198 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-04-26T04:32:48.225 INFO:tasks.rados.rados.0.smithi178.stdout:180: writing smithi17829308-31 from 4276224 to 4947968 tid 1 2024-04-26T04:32:48.232 INFO:tasks.rados.rados.0.smithi178.stdout:180: writing smithi17829308-31 from 4947968 to 5718016 tid 2 2024-04-26T04:32:48.232 INFO:tasks.rados.rados.0.smithi178.stdout:181: read oid 49 snap -1 2024-04-26T04:32:48.232 INFO:tasks.rados.rados.0.smithi178.stdout:181: expect (ObjNum 67 snap 3 seq_num 67) 2024-04-26T04:32:48.233 INFO:tasks.rados.rados.0.smithi178.stdout:182: read oid 38 snap -1 2024-04-26T04:32:48.233 INFO:tasks.rados.rados.0.smithi178.stdout:182: expect (ObjNum 3 snap 0 seq_num 3) 2024-04-26T04:32:48.233 INFO:tasks.rados.rados.0.smithi178.stdout:183: read oid 40 snap -1 2024-04-26T04:32:48.233 INFO:tasks.rados.rados.0.smithi178.stdout:183: expect (ObjNum 56 snap 1 seq_num 56) 2024-04-26T04:32:48.233 INFO:tasks.rados.rados.0.smithi178.stdout:184: snap_create 2024-04-26T04:32:48.234 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:48.232+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:48.315 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:48.315 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:48.329 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T04:32:48.334 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:48.333+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:48.435 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:48.434+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:48.483 INFO:tasks.rados.rados.0.smithi178.stdout:180: finishing write tid 1 to smithi17829308-31 2024-04-26T04:32:48.483 INFO:tasks.rados.rados.0.smithi178.stdout:180: finishing write tid 2 to smithi17829308-31 2024-04-26T04:32:48.494 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:48.494 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:48.508 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T04:32:48.536 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:48.533+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:48.594 INFO:tasks.rados.rados.0.smithi178.stdout:179: expect (ObjNum 79 snap 8 seq_num 79) 2024-04-26T04:32:48.625 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:48.625 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:48.636 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:48.633+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:48.641 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T04:32:48.673 INFO:tasks.rados.rados.0.smithi178.stdout:180: finishing write tid 3 to smithi17829308-31 2024-04-26T04:32:48.673 INFO:tasks.rados.rados.0.smithi178.stdout:182: expect (ObjNum 3 snap 0 seq_num 3) 2024-04-26T04:32:48.675 INFO:tasks.rados.rados.0.smithi178.stdout:181: expect (ObjNum 67 snap 3 seq_num 67) 2024-04-26T04:32:48.738 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:48.736+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:48.743 INFO:tasks.rados.rados.0.smithi178.stdout:183: expect (ObjNum 56 snap 1 seq_num 56) 2024-04-26T04:32:48.762 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:48.762 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:48.776 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T04:32:48.839 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:48.838+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:48.941 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:48.936+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:48.942 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:48.942 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:48.943 INFO:tasks.rados.rados.0.smithi178.stdout:180: finishing write tid 4 to smithi17829308-31 2024-04-26T04:32:48.943 INFO:tasks.rados.rados.0.smithi178.stdout:180: oid 31 updating version 0 to 36 2024-04-26T04:32:48.943 INFO:tasks.rados.rados.0.smithi178.stdout:180: oid 31 updating version 36 to 38 2024-04-26T04:32:48.943 INFO:tasks.rados.rados.0.smithi178.stdout:180: oid 31 version 38 is already newer than 37 2024-04-26T04:32:48.943 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 31 v 38 (ObjNum 85 snap 11 seq_num 85) dirty exists 2024-04-26T04:32:48.943 INFO:tasks.rados.rados.0.smithi178.stdout:180: left oid 31 (ObjNum 85 snap 11 seq_num 85) 2024-04-26T04:32:48.956 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T04:32:49.046 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:49.044+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:49.121 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:49.121 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:49.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T04:32:49.142 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:49.141+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:49.174 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.11 to 1 (10000) 2024-04-26T04:32:49.180 INFO:tasks.rados.rados.0.smithi178.stdout:179: done (5 left) 2024-04-26T04:32:49.180 INFO:tasks.rados.rados.0.smithi178.stdout:180: done (4 left) 2024-04-26T04:32:49.180 INFO:tasks.rados.rados.0.smithi178.stdout:181: done (3 left) 2024-04-26T04:32:49.180 INFO:tasks.rados.rados.0.smithi178.stdout:182: done (2 left) 2024-04-26T04:32:49.180 INFO:tasks.rados.rados.0.smithi178.stdout:183: done (1 left) 2024-04-26T04:32:49.181 INFO:tasks.rados.rados.0.smithi178.stdout:184: done (0 left) 2024-04-26T04:32:49.181 INFO:tasks.rados.rados.0.smithi178.stdout:185: snap_remove snap 11 2024-04-26T04:32:49.201 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-04-26T04:32:49.230 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-26T04:32:49.244 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:49.241+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:49.296 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:49.297 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:49.313 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T04:32:49.344 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:49.341+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:49.437 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:49.437 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:49.445 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:49.440+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:49.451 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T04:32:49.458 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T04:32:49.546 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:49.545+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:49.620 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:49.620 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:49.634 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T04:32:49.647 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:49.645+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:49.735 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:49.736 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:49.749 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:49.748+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:49.767 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:49.767 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:49.768 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:49.768 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:49.782 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T04:32:49.800 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:49.801 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:49.830 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:49.830 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:49.850 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:49.849+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:49.859 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:49.859 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:49.890 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:49.890 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:49.901 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:49.901 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:49.901 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:32:49.901 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:32:49.901 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:49.912 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:49.912 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:49.917 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T04:32:49.940 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:49.940 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:49.954 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:49.948+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:49.965 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:49.965 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:49.991 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:49.992 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:50.019 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:50.019 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:50.044 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:50.044 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:32:50.044 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:32:50.044 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:32:50.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.044 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:263 3.as0 3:51a91c23:::smithi17829308-18:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:50.044 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.817670+0000", 2024-04-26T04:32:50.044 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 20.22474665, 2024-04-26T04:32:50.044 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30863979699999999, 2024-04-26T04:32:50.044 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.045 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.045 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:50.045 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.045 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.045 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.045 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 263 2024-04-26T04:32:50.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.045 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.045 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.045 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.045 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817670+0000", 2024-04-26T04:32:50.045 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817670+0000", 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817673+0000", 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9560000000000002e-06 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817687+0000", 2024-04-26T04:32:50.046 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3923e-05 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817719+0000", 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2231999999999999e-05 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817738+0000", 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8473000000000001e-05 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.047 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817834+0000", 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.5913000000000001e-05 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817971+0000", 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013697899999999999 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.818954+0000", 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00098307899999999998 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.048 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.924736+0000", 2024-04-26T04:32:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10578224899999999 2024-04-26T04:32:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.126181+0000", 2024-04-26T04:32:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20144535999999999 2024-04-26T04:32:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.126310+0000", 2024-04-26T04:32:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012863300000000001 2024-04-26T04:32:50.049 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:264 3.as0 3:51a91c23:::smithi17829308-18:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.817774+0000", 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 20.224642824, 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30889035999999997, 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.050 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 264 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817774+0000", 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817774+0000", 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817775+0000", 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8299999999999995e-07 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817781+0000", 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9270000000000002e-06 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817793+0000", 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2113000000000001e-05 2024-04-26T04:32:50.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817802+0000", 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8000000000000004e-06 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819893+0000", 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0020906169999999999 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.820146+0000", 2024-04-26T04:32:50.053 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000253415 2024-04-26T04:32:50.054 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.054 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.054 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.054 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.821588+0000", 2024-04-26T04:32:50.054 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001441821 2024-04-26T04:32:50.054 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.054 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.054 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.054 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.924739+0000", 2024-04-26T04:32:50.054 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103150835 2024-04-26T04:32:50.054 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.054 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.054 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.126551+0000", 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20181190500000001 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.126665+0000", 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011394400000000001 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:265 3.as0 3:51a91c23:::smithi17829308-18:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:50.055 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.817865+0000", 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 20.22455179, 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50897067299999998, 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 265 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817865+0000", 2024-04-26T04:32:50.056 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.057 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.057 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.057 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817865+0000", 2024-04-26T04:32:50.057 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.057 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.057 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.057 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817867+0000", 2024-04-26T04:32:50.057 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0999999999999998e-06 2024-04-26T04:32:50.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.057 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.057 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.058 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817876+0000", 2024-04-26T04:32:50.058 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8060000000000001e-06 2024-04-26T04:32:50.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.058 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.058 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.058 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817899+0000", 2024-04-26T04:32:50.058 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2716000000000001e-05 2024-04-26T04:32:50.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.058 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.058 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.058 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817910+0000", 2024-04-26T04:32:50.058 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1579999999999999e-05 2024-04-26T04:32:50.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.059 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.059 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.822433+0000", 2024-04-26T04:32:50.059 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0045222869999999998 2024-04-26T04:32:50.059 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.059 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.059 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.822747+0000", 2024-04-26T04:32:50.059 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00031401299999999998 2024-04-26T04:32:50.059 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.059 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.059 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.823790+0000", 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00104357 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.924740+0000", 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100949605 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.326718+0000", 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40197800900000002 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.060 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.326836+0000", 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011798699999999999 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 36/30 ECSubWriteReply(tid=17, last_complete=36'10, committed=1, applied=1))", 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:30.026129+0000", 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 20.016288450000001, 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30150891400000002, 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.061 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.026129+0000", 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.026129+0000", 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.062 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.026130+0000", 2024-04-26T04:32:50.063 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.097e-06 2024-04-26T04:32:50.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.063 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.063 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.026138+0000", 2024-04-26T04:32:50.063 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.0700000000000007e-06 2024-04-26T04:32:50.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.063 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.063 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.026158+0000", 2024-04-26T04:32:50.063 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0522000000000001e-05 2024-04-26T04:32:50.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.026169+0000", 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1060000000000001e-05 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.326593+0000", 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.300423251 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327637+0000", 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0010449140000000001 2024-04-26T04:32:50.064 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:328 3.as0 3:527650bd:::smithi17829308-27:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.408149+0000", 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.634268457999999, 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.409867282, 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.065 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 328 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408149+0000", 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408149+0000", 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.066 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408152+0000", 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.8870000000000002e-06 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408172+0000", 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.912e-05 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408216+0000", 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.4370999999999999e-05 2024-04-26T04:32:50.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408238+0000", 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1820000000000001e-05 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408355+0000", 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000116882 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408364+0000", 2024-04-26T04:32:50.068 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.4099999999999997e-06 2024-04-26T04:32:50.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.069 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.069 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.069 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.412080+0000", 2024-04-26T04:32:50.069 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.003715566 2024-04-26T04:32:50.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.069 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.069 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.069 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.412245+0000", 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000165914 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413482+0000", 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0012361690000000001 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616957+0000", 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20347567 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.070 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.817777+0000", 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20081958 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.818016+0000", 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000238893 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:329 3.as0 3:527650bd:::smithi17829308-27:head [rollback 2] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:50.071 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.408276+0000", 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.634140554, 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61054785099999997, 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 329 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.072 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408276+0000", 2024-04-26T04:32:50.073 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.073 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.073 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.073 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408276+0000", 2024-04-26T04:32:50.073 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.073 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.073 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.073 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408277+0000", 2024-04-26T04:32:50.073 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.7999999999999993e-07 2024-04-26T04:32:50.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.073 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.074 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.074 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408284+0000", 2024-04-26T04:32:50.074 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.0389999999999999e-06 2024-04-26T04:32:50.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.074 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.074 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.074 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408311+0000", 2024-04-26T04:32:50.074 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6452999999999999e-05 2024-04-26T04:32:50.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.074 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.074 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.074 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408324+0000", 2024-04-26T04:32:50.075 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3259e-05 2024-04-26T04:32:50.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.075 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.075 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408439+0000", 2024-04-26T04:32:50.075 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000114789 2024-04-26T04:32:50.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.075 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:50.076 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408444+0000", 2024-04-26T04:32:50.076 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.3700000000000003e-06 2024-04-26T04:32:50.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.076 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.076 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.414140+0000", 2024-04-26T04:32:50.076 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0056952110000000004 2024-04-26T04:32:50.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.076 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.076 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.414263+0000", 2024-04-26T04:32:50.077 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012392699999999999 2024-04-26T04:32:50.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.077 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.077 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.415016+0000", 2024-04-26T04:32:50.077 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00075301600000000002 2024-04-26T04:32:50.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.077 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.077 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616959+0000", 2024-04-26T04:32:50.078 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20194278800000001 2024-04-26T04:32:50.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.078 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:50.078 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018750+0000", 2024-04-26T04:32:50.078 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40179092999999999 2024-04-26T04:32:50.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.079 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.079 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018824+0000", 2024-04-26T04:32:50.079 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.4089000000000002e-05 2024-04-26T04:32:50.079 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.079 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.079 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.079 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:330 3.as0 3:527650bd:::smithi17829308-27:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:50.079 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.408341+0000", 2024-04-26T04:32:50.079 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.63407572, 2024-04-26T04:32:50.080 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61070586699999996, 2024-04-26T04:32:50.080 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.080 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.080 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:50.080 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.080 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.080 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.080 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 330 2024-04-26T04:32:50.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.080 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.080 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.081 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408341+0000", 2024-04-26T04:32:50.081 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.081 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.081 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408341+0000", 2024-04-26T04:32:50.081 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.081 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.081 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408342+0000", 2024-04-26T04:32:50.081 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.6700000000000003e-07 2024-04-26T04:32:50.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.082 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.082 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408346+0000", 2024-04-26T04:32:50.082 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7299999999999999e-06 2024-04-26T04:32:50.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.082 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.082 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408357+0000", 2024-04-26T04:32:50.082 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1276e-05 2024-04-26T04:32:50.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408371+0000", 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3763e-05 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408472+0000", 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010149600000000001 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408477+0000", 2024-04-26T04:32:50.083 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5029999999999999e-06 2024-04-26T04:32:50.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.084 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.084 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.515978+0000", 2024-04-26T04:32:50.084 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.107501546 2024-04-26T04:32:50.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.084 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.084 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.516082+0000", 2024-04-26T04:32:50.084 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010331 2024-04-26T04:32:50.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.084 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.516573+0000", 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000491316 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.617004+0000", 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100431009 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018996+0000", 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40199249199999998 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.086 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.086 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.019047+0000", 2024-04-26T04:32:50.086 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0658999999999997e-05 2024-04-26T04:32:50.086 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.086 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.086 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.086 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=26, last_complete=40'18, committed=1, applied=1))", 2024-04-26T04:32:50.086 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.418140+0000", 2024-04-26T04:32:50.086 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.624276521000001, 2024-04-26T04:32:50.086 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29900827400000002, 2024-04-26T04:32:50.086 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.418140+0000", 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.418140+0000", 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.088 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.088 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.418141+0000", 2024-04-26T04:32:50.088 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.4399999999999999e-07 2024-04-26T04:32:50.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.088 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.088 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.418145+0000", 2024-04-26T04:32:50.088 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.9049999999999999e-06 2024-04-26T04:32:50.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.088 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.088 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.418158+0000", 2024-04-26T04:32:50.088 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2493999999999999e-05 2024-04-26T04:32:50.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.089 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.089 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518257+0000", 2024-04-26T04:32:50.089 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100099681 2024-04-26T04:32:50.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.089 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.089 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.717115+0000", 2024-04-26T04:32:50.089 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.198858018 2024-04-26T04:32:50.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.090 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.090 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.717149+0000", 2024-04-26T04:32:50.090 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3432000000000001e-05 2024-04-26T04:32:50.090 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.090 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.090 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.090 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=27, last_complete=40'19, committed=1, applied=1))", 2024-04-26T04:32:50.090 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.419714+0000", 2024-04-26T04:32:50.090 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.622702908000001, 2024-04-26T04:32:50.090 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29753107699999998, 2024-04-26T04:32:50.090 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.091 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.091 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.091 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.092 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.419714+0000", 2024-04-26T04:32:50.092 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.092 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.093 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.419714+0000", 2024-04-26T04:32:50.093 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.093 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.093 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.419715+0000", 2024-04-26T04:32:50.093 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.2600000000000001e-07 2024-04-26T04:32:50.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.093 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.093 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.419720+0000", 2024-04-26T04:32:50.093 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.5269999999999998e-06 2024-04-26T04:32:50.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.094 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.094 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.419738+0000", 2024-04-26T04:32:50.094 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7139e-05 2024-04-26T04:32:50.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.094 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.094 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.419748+0000", 2024-04-26T04:32:50.094 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0343000000000001e-05 2024-04-26T04:32:50.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.095 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.095 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.717211+0000", 2024-04-26T04:32:50.095 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29746346299999998 2024-04-26T04:32:50.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.095 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.096 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.717245+0000", 2024-04-26T04:32:50.096 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3779000000000003e-05 2024-04-26T04:32:50.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.096 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.096 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 39/30 ECSubWriteReply(tid=27, last_complete=40'19, committed=1, applied=1))", 2024-04-26T04:32:50.096 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.420941+0000", 2024-04-26T04:32:50.096 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.621476335000001, 2024-04-26T04:32:50.096 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29637106299999999, 2024-04-26T04:32:50.096 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.097 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.097 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.097 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.097 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.097 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.420941+0000", 2024-04-26T04:32:50.097 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.097 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.097 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.420941+0000", 2024-04-26T04:32:50.097 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.420942+0000", 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6500000000000008e-07 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.420948+0000", 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.1680000000000001e-06 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.420968+0000", 2024-04-26T04:32:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0415e-05 2024-04-26T04:32:50.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.099 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.099 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.420979+0000", 2024-04-26T04:32:50.099 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0353000000000001e-05 2024-04-26T04:32:50.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.099 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.099 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.717283+0000", 2024-04-26T04:32:50.099 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29630447599999998 2024-04-26T04:32:50.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.100 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.100 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.717312+0000", 2024-04-26T04:32:50.100 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8685999999999999e-05 2024-04-26T04:32:50.100 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.100 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.100 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.100 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=27, last_complete=40'19, committed=1, applied=1))", 2024-04-26T04:32:50.100 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.518317+0000", 2024-04-26T04:32:50.100 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.524099704999999, 2024-04-26T04:32:50.100 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39993706099999998, 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518317+0000", 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518317+0000", 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518319+0000", 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7099999999999999e-06 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518329+0000", 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0159999999999999e-05 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518362+0000", 2024-04-26T04:32:50.102 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3321999999999997e-05 2024-04-26T04:32:50.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.103 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.103 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518380+0000", 2024-04-26T04:32:50.103 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7693000000000002e-05 2024-04-26T04:32:50.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.103 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.103 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.918198+0000", 2024-04-26T04:32:50.103 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.399817537 2024-04-26T04:32:50.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.104 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.104 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.918254+0000", 2024-04-26T04:32:50.104 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.6638999999999998e-05 2024-04-26T04:32:50.104 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.104 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.104 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.104 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:32:50.104 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.518641+0000", 2024-04-26T04:32:50.104 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.523775594, 2024-04-26T04:32:50.104 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39974739300000001, 2024-04-26T04:32:50.104 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518641+0000", 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518641+0000", 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.106 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.106 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518643+0000", 2024-04-26T04:32:50.106 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9659999999999999e-06 2024-04-26T04:32:50.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.106 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.106 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518656+0000", 2024-04-26T04:32:50.106 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2945999999999999e-05 2024-04-26T04:32:50.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.106 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518692+0000", 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.5485999999999999e-05 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518716+0000", 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3906000000000001e-05 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.918350+0000", 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39963400700000001 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.918389+0000", 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.9082000000000003e-05 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.520051+0000", 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.522365595, 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49885426900000002, 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.108 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.109 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.109 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.109 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.109 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520051+0000", 2024-04-26T04:32:50.109 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.109 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.109 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520051+0000", 2024-04-26T04:32:50.109 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.110 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.110 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520052+0000", 2024-04-26T04:32:50.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.5000000000000001e-07 2024-04-26T04:32:50.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.110 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.110 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520062+0000", 2024-04-26T04:32:50.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9799999999999993e-06 2024-04-26T04:32:50.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.110 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520079+0000", 2024-04-26T04:32:50.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6518999999999999e-05 2024-04-26T04:32:50.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.111 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520087+0000", 2024-04-26T04:32:50.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.5329999999999994e-06 2024-04-26T04:32:50.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.111 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018885+0000", 2024-04-26T04:32:50.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49879782299999997 2024-04-26T04:32:50.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.112 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.112 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018906+0000", 2024-04-26T04:32:50.112 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0564000000000002e-05 2024-04-26T04:32:50.112 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.112 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.112 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.112 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:32:50.112 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.522047+0000", 2024-04-26T04:32:50.112 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.520369909999999, 2024-04-26T04:32:50.112 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49659926700000001, 2024-04-26T04:32:50.113 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.113 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.113 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.113 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.113 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.113 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522047+0000", 2024-04-26T04:32:50.113 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.114 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.114 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522047+0000", 2024-04-26T04:32:50.114 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.114 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.114 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522048+0000", 2024-04-26T04:32:50.114 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1200000000000001e-07 2024-04-26T04:32:50.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.114 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.114 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522054+0000", 2024-04-26T04:32:50.115 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.049e-06 2024-04-26T04:32:50.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.115 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.115 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522070+0000", 2024-04-26T04:32:50.115 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5492999999999999e-05 2024-04-26T04:32:50.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.115 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.115 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522079+0000", 2024-04-26T04:32:50.115 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1679999999999993e-06 2024-04-26T04:32:50.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018623+0000", 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49654389700000001 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018646+0000", 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3748e-05 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.116 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=27, last_complete=40'19, committed=1, applied=1))", 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.616517+0000", 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.425900065, 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40232084899999998, 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616517+0000", 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.117 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.118 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616517+0000", 2024-04-26T04:32:50.118 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.118 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.118 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616518+0000", 2024-04-26T04:32:50.118 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.0999999999999997e-07 2024-04-26T04:32:50.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.118 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.118 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616523+0000", 2024-04-26T04:32:50.118 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0490000000000003e-06 2024-04-26T04:32:50.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616539+0000", 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6555999999999999e-05 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616549+0000", 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8269999999999993e-06 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018674+0000", 2024-04-26T04:32:50.119 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40212457600000001 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018838+0000", 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000164031 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.616951+0000", 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.425466296, 2024-04-26T04:32:50.120 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.402569169, 2024-04-26T04:32:50.121 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.121 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.121 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.121 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.121 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.121 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616951+0000", 2024-04-26T04:32:50.121 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.121 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.121 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616951+0000", 2024-04-26T04:32:50.121 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.122 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.122 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616951+0000", 2024-04-26T04:32:50.122 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.8700000000000005e-07 2024-04-26T04:32:50.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.122 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.122 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616957+0000", 2024-04-26T04:32:50.122 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.3589999999999999e-06 2024-04-26T04:32:50.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.122 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.123 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616974+0000", 2024-04-26T04:32:50.123 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7278999999999998e-05 2024-04-26T04:32:50.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.123 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.123 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616983+0000", 2024-04-26T04:32:50.123 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9500000000000007e-06 2024-04-26T04:32:50.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.123 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.123 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018945+0000", 2024-04-26T04:32:50.123 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40196159399999998 2024-04-26T04:32:50.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.124 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.124 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.019520+0000", 2024-04-26T04:32:50.124 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0005752 2024-04-26T04:32:50.124 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.124 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.124 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.124 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:32:50.124 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.620433+0000", 2024-04-26T04:32:50.124 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.421984408, 2024-04-26T04:32:50.125 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29806305700000002, 2024-04-26T04:32:50.125 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.125 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.125 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.125 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.125 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.125 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.620433+0000", 2024-04-26T04:32:50.125 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.125 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.126 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.620433+0000", 2024-04-26T04:32:50.126 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.126 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.126 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.620433+0000", 2024-04-26T04:32:50.126 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5799999999999998e-07 2024-04-26T04:32:50.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.126 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.126 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.620439+0000", 2024-04-26T04:32:50.126 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.1900000000000003e-06 2024-04-26T04:32:50.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.127 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.127 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.127 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.620451+0000", 2024-04-26T04:32:50.127 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2103000000000001e-05 2024-04-26T04:32:50.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.127 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.127 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.127 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.620457+0000", 2024-04-26T04:32:50.127 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.832e-06 2024-04-26T04:32:50.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.127 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.128 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.128 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.918467+0000", 2024-04-26T04:32:50.128 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29800976499999998 2024-04-26T04:32:50.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.128 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.128 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.128 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.918496+0000", 2024-04-26T04:32:50.128 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8408999999999999e-05 2024-04-26T04:32:50.128 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.128 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.128 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.128 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:333 3.as0 3:527650bd:::smithi17829308-27:head [copy-from ver 2] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.120049+0000", 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 14.922368175999999, 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.66333988200000005, 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 333 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.129 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.130 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.130 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120049+0000", 2024-04-26T04:32:50.130 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.130 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.130 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.130 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.130 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120049+0000", 2024-04-26T04:32:50.130 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.130 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.130 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.130 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.130 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120050+0000", 2024-04-26T04:32:50.130 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.536e-06 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120057+0000", 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6869999999999997e-06 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120074+0000", 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6787e-05 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.131 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120086+0000", 2024-04-26T04:32:50.132 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2665e-05 2024-04-26T04:32:50.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.132 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.132 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.132 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.220306+0000", 2024-04-26T04:32:50.132 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10021996900000001 2024-04-26T04:32:50.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.132 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.132 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.132 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.220485+0000", 2024-04-26T04:32:50.132 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017836800000000001 2024-04-26T04:32:50.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.577412+0000", 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.35692740299999998 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.580481+0000", 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.003068735 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.783160+0000", 2024-04-26T04:32:50.133 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20267940200000001 2024-04-26T04:32:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.783389+0000", 2024-04-26T04:32:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022833 2024-04-26T04:32:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:334 3.as0 3:527650bd:::smithi17829308-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.120106+0000", 2024-04-26T04:32:50.134 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 14.922310693, 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.459292215, 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 334 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120106+0000", 2024-04-26T04:32:50.135 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.136 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.136 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120106+0000", 2024-04-26T04:32:50.136 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.136 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.136 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120107+0000", 2024-04-26T04:32:50.136 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.540000000000001e-07 2024-04-26T04:32:50.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.137 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.137 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120111+0000", 2024-04-26T04:32:50.137 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.6160000000000002e-06 2024-04-26T04:32:50.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.137 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.137 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.137 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120124+0000", 2024-04-26T04:32:50.137 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2965e-05 2024-04-26T04:32:50.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.137 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.137 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.137 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.220210+0000", 2024-04-26T04:32:50.137 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100086047 2024-04-26T04:32:50.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.138 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.138 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221197+0000", 2024-04-26T04:32:50.138 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00098664200000000003 2024-04-26T04:32:50.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.138 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:32:50.138 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221256+0000", 2024-04-26T04:32:50.138 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9734000000000001e-05 2024-04-26T04:32:50.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.138 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.578868+0000", 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.35761166799999999 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579093+0000", 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022499099999999999 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579399+0000", 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00030559800000000002 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.139 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:387 3.as0 3:51a91c23:::smithi17829308-18:head [delete] snapc 8=[7,6,5] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e46)", 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:40.371914+0000", 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 9.6705034130000005, 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.207759572, 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.140 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 387 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.371914+0000", 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.371914+0000", 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.141 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.371917+0000", 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.45e-06 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.371929+0000", 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2133000000000001e-05 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.371962+0000", 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2515999999999998e-05 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.371980+0000", 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8598999999999999e-05 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.472199+0000", 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10021858 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.472212+0000", 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3273e-05 2024-04-26T04:32:50.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.144 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.144 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.144 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.472911+0000", 2024-04-26T04:32:50.144 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00069932600000000001 2024-04-26T04:32:50.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.144 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.144 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.144 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.473084+0000", 2024-04-26T04:32:50.144 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017290900000000001 2024-04-26T04:32:50.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.144 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.144 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.144 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.474363+0000", 2024-04-26T04:32:50.145 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001278239 2024-04-26T04:32:50.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.145 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.145 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.476906+0000", 2024-04-26T04:32:50.145 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0025430029999999998 2024-04-26T04:32:50.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.145 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:50.145 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.579545+0000", 2024-04-26T04:32:50.145 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.102639177 2024-04-26T04:32:50.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.146 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.146 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:40.579673+0000", 2024-04-26T04:32:50.146 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000128367 2024-04-26T04:32:50.146 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.146 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.146 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.146 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.146 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.146 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:50.147 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:50.148 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:50.148 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:50.149 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:50.149 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T04:32:50.150 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:50.150 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:50.150 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:50.150 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:50.151 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:50.151 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:50.152 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:50.144+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-04-26T04:32:50.180 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.12 to 1 (10000) 2024-04-26T04:32:50.185 INFO:tasks.rados.rados.0.smithi178.stdout:185: done (0 left) 2024-04-26T04:32:50.186 INFO:tasks.rados.rados.0.smithi178.stdout:186: read oid 36 snap -1 2024-04-26T04:32:50.186 INFO:tasks.rados.rados.0.smithi178.stdout:186: expect (ObjNum 35 snap 0 seq_num 35) 2024-04-26T04:32:50.186 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 1 current snap is 12 2024-04-26T04:32:50.186 INFO:tasks.rados.rados.0.smithi178.stdout:187: seq_num 86 ranges {0=688128,688128=540672,1228800=229376} 2024-04-26T04:32:50.190 INFO:tasks.rados.rados.0.smithi178.stdout:187: writing smithi17829308-1 from 0 to 688128 tid 1 2024-04-26T04:32:50.194 INFO:tasks.rados.rados.0.smithi178.stdout:187: writing smithi17829308-1 from 688128 to 1228800 tid 2 2024-04-26T04:32:50.196 INFO:tasks.rados.rados.0.smithi178.stdout:187: writing smithi17829308-1 from 1228800 to 1458176 tid 3 2024-04-26T04:32:50.196 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 9 current snap is 12 2024-04-26T04:32:50.196 INFO:tasks.rados.rados.0.smithi178.stdout:188: seq_num 87 ranges {1212416=704512,1916928=688128,2605056=81920} 2024-04-26T04:32:50.204 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-04-26T04:32:50.210 INFO:tasks.rados.rados.0.smithi178.stdout:188: writing smithi17829308-9 from 1212416 to 1916928 tid 1 2024-04-26T04:32:50.218 INFO:tasks.rados.rados.0.smithi178.stdout:188: writing smithi17829308-9 from 1916928 to 2605056 tid 2 2024-04-26T04:32:50.257 INFO:tasks.rados.rados.0.smithi178.stdout:188: writing smithi17829308-9 from 2605056 to 2686976 tid 3 2024-04-26T04:32:50.257 INFO:tasks.rados.rados.0.smithi178.stdout:189: rmattr oid 11 current snap is 12 2024-04-26T04:32:50.257 INFO:tasks.rados.rados.0.smithi178.stdout:189: done (3 left) 2024-04-26T04:32:50.257 INFO:tasks.rados.rados.0.smithi178.stdout:190: snap_remove snap 10 2024-04-26T04:32:50.257 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:50.254+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:50.278 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:50.278 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:50.278 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 0 2024-04-26T04:32:50.278 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:50.293 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T04:32:50.357 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:50.353+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:50.458 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:50.456+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:50.459 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:50.460 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:50.460 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:32:50.460 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:32:50.460 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:50.474 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T04:32:50.559 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:50.558+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:50.603 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:50.603 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:32:50.603 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:32:50.603 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:32:50.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 36/30 ECSubWrite(tid=29, reqid=client.4670.0:266, at_version=36'17, trim_to=0'0, roll_forward_to=35'16))", 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.819415+0000", 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 20.780863881999998, 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20185976, 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819415+0000", 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.605 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.605 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819415+0000", 2024-04-26T04:32:50.605 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.605 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.605 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819416+0000", 2024-04-26T04:32:50.605 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.6000000000000004e-07 2024-04-26T04:32:50.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.605 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.605 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819506+0000", 2024-04-26T04:32:50.606 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.0537000000000004e-05 2024-04-26T04:32:50.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.606 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.606 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819532+0000", 2024-04-26T04:32:50.606 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5709000000000001e-05 2024-04-26T04:32:50.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.606 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.606 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819552+0000", 2024-04-26T04:32:50.606 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0373e-05 2024-04-26T04:32:50.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.019889+0000", 2024-04-26T04:32:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20033689900000001 2024-04-26T04:32:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.019905+0000", 2024-04-26T04:32:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5571999999999999e-05 2024-04-26T04:32:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.607 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.021199+0000", 2024-04-26T04:32:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001294623 2024-04-26T04:32:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.021275+0000", 2024-04-26T04:32:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5487000000000004e-05 2024-04-26T04:32:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.as1 36/30 ECSubWrite(tid=18, reqid=client.4670.0:265, at_version=0'0, trim_to=0'0, roll_forward_to=36'10))", 2024-04-26T04:32:50.608 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:30.327162+0000", 2024-04-26T04:32:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 20.273116468000001, 2024-04-26T04:32:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20162941100000001, 2024-04-26T04:32:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327162+0000", 2024-04-26T04:32:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.609 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327162+0000", 2024-04-26T04:32:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327165+0000", 2024-04-26T04:32:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1639999999999999e-06 2024-04-26T04:32:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327194+0000", 2024-04-26T04:32:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9388999999999999e-05 2024-04-26T04:32:50.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327237+0000", 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3065000000000002e-05 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.427384+0000", 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100147443 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.527653+0000", 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100268697 2024-04-26T04:32:50.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.527668+0000", 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4449e-05 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.528724+0000", 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0010562869999999999 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.528792+0000", 2024-04-26T04:32:50.612 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7916999999999994e-05 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 39/30 ECSubWriteReply(tid=26, last_complete=39'15, committed=1, applied=1))", 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.010719+0000", 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.589559537, 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.200420977, 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010719+0000", 2024-04-26T04:32:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010719+0000", 2024-04-26T04:32:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.614 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010721+0000", 2024-04-26T04:32:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.198e-06 2024-04-26T04:32:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010728+0000", 2024-04-26T04:32:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.2219999999999996e-06 2024-04-26T04:32:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.010745+0000", 2024-04-26T04:32:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7180000000000002e-05 2024-04-26T04:32:50.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.110846+0000", 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10010114000000001 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.211083+0000", 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100237346 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.211140+0000", 2024-04-26T04:32:50.616 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.6891000000000001e-05 2024-04-26T04:32:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:327 3.ds0 3:b7d7af40:::smithi17829308-48:2 [read 0~802816,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.408060+0000", 2024-04-26T04:32:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.192218748999998, 2024-04-26T04:32:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21258864, 2024-04-26T04:32:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.617 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 327 2024-04-26T04:32:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408060+0000", 2024-04-26T04:32:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.618 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.619 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408060+0000", 2024-04-26T04:32:50.619 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.619 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.619 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408061+0000", 2024-04-26T04:32:50.619 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.7999999999999995e-07 2024-04-26T04:32:50.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.619 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.619 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408065+0000", 2024-04-26T04:32:50.619 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.25e-06 2024-04-26T04:32:50.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408074+0000", 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1949999999999992e-06 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408081+0000", 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.2590000000000003e-06 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408199+0000", 2024-04-26T04:32:50.620 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000118066 2024-04-26T04:32:50.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.621 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.621 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:50.621 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408204+0000", 2024-04-26T04:32:50.621 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0289999999999996e-06 2024-04-26T04:32:50.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.621 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.621 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.621 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.513122+0000", 2024-04-26T04:32:50.621 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.104917964 2024-04-26T04:32:50.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.621 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.622 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.622 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.513302+0000", 2024-04-26T04:32:50.622 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000179092 2024-04-26T04:32:50.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.622 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.622 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.620649+0000", 2024-04-26T04:32:50.622 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.107347205 2024-04-26T04:32:50.622 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.622 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.622 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.623 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.bs5 40/30 ECSubWrite(tid=106, reqid=client.4670.0:322, at_version=40'19, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:32:50.623 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.611673+0000", 2024-04-26T04:32:50.623 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.988605502, 2024-04-26T04:32:50.623 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20070944199999999, 2024-04-26T04:32:50.623 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.623 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.623 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.623 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.623 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.624 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.611673+0000", 2024-04-26T04:32:50.624 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.624 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.624 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.624 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.624 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.611673+0000", 2024-04-26T04:32:50.625 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.625 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.625 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.625 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.625 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.611674+0000", 2024-04-26T04:32:50.625 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8100000000000001e-07 2024-04-26T04:32:50.625 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.625 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.625 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.625 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.611815+0000", 2024-04-26T04:32:50.626 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014082299999999999 2024-04-26T04:32:50.626 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.626 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.626 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.611830+0000", 2024-04-26T04:32:50.626 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4555999999999999e-05 2024-04-26T04:32:50.626 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.626 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.626 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.711943+0000", 2024-04-26T04:32:50.626 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100112942 2024-04-26T04:32:50.626 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.626 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.712265+0000", 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000322188 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.712273+0000", 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.0020000000000006e-06 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.812299+0000", 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100025989 2024-04-26T04:32:50.627 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.812383+0000", 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4060999999999994e-05 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:381 3.ds0 3:b7d7af40:::smithi17829308-48:5 [read 0~802816,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.431434+0000", 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 11.168844834, 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40271601499999998, 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 381 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431434+0000", 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431434+0000", 2024-04-26T04:32:50.629 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431436+0000", 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3140000000000002e-06 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431447+0000", 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0655e-05 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.630 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431471+0000", 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4470000000000001e-05 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431488+0000", 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6129000000000001e-05 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.494163+0000", 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.062675393999999995 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.631 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.494174+0000", 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.114e-05 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.497530+0000", 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0033555439999999998 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.497684+0000", 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015432600000000001 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.633 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.633 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.834150+0000", 2024-04-26T04:32:50.633 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.33646604299999999 2024-04-26T04:32:50.633 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.633 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.633 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.633 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.633 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.633 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:382 3.ds0 3:b7d7af40:::smithi17829308-48:5 [read 0~802816,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:32:50.633 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:32:50.633 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 11.168766226000001, 2024-04-26T04:32:50.633 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50469882499999996, 2024-04-26T04:32:50.633 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.634 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.634 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:50.634 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.634 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.634 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.634 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 382 2024-04-26T04:32:50.634 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.634 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.634 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.634 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.634 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:32:50.635 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.635 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.635 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.635 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:32:50.635 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.635 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.637 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.638 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:32:50.638 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9800000000000003e-07 2024-04-26T04:32:50.638 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.638 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.638 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.638 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431521+0000", 2024-04-26T04:32:50.638 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7449999999999995e-06 2024-04-26T04:32:50.638 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.638 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.638 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.638 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431529+0000", 2024-04-26T04:32:50.638 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.3599999999999996e-06 2024-04-26T04:32:50.638 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531644+0000", 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100114514 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.699490+0000", 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.16784555000000001 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.699653+0000", 2024-04-26T04:32:50.639 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016372199999999999 2024-04-26T04:32:50.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.640 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.640 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.640 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.936211+0000", 2024-04-26T04:32:50.640 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23655823600000001 2024-04-26T04:32:50.640 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.640 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.640 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.640 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.640 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:383 3.ds0 3:b7d7af40:::smithi17829308-48:5 [read 0~802816,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:32:50.640 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.531687+0000", 2024-04-26T04:32:50.640 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 11.068591693, 2024-04-26T04:32:50.641 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.405754104, 2024-04-26T04:32:50.641 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.641 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.641 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:50.641 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.641 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.641 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.641 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 383 2024-04-26T04:32:50.641 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.641 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.641 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.641 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.641 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531687+0000", 2024-04-26T04:32:50.642 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.642 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.642 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.642 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531687+0000", 2024-04-26T04:32:50.642 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.642 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.642 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.642 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531689+0000", 2024-04-26T04:32:50.642 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.063e-06 2024-04-26T04:32:50.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.642 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531702+0000", 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3156000000000001e-05 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531733+0000", 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0335999999999999e-05 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531766+0000", 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3623e-05 2024-04-26T04:32:50.643 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.700194+0000", 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.16842744400000001 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.700343+0000", 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000148763 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.937441+0000", 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23709871900000001 2024-04-26T04:32:50.644 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.645 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.645 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.645 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.645 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.645 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpReadReply(3.ds0 44/30 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-26T04:32:50.645 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.632626+0000", 2024-04-26T04:32:50.645 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 10.967652719, 2024-04-26T04:32:50.645 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20157424099999999, 2024-04-26T04:32:50.645 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.645 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.645 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.645 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.645 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.632626+0000", 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.632626+0000", 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.632628+0000", 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6160000000000001e-06 2024-04-26T04:32:50.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.647 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.647 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.632812+0000", 2024-04-26T04:32:50.647 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018405900000000001 2024-04-26T04:32:50.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.647 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.647 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.632841+0000", 2024-04-26T04:32:50.647 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9051999999999999e-05 2024-04-26T04:32:50.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.647 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.647 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.732979+0000", 2024-04-26T04:32:50.648 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100138158 2024-04-26T04:32:50.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.648 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.648 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.648 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.833242+0000", 2024-04-26T04:32:50.648 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10026294500000001 2024-04-26T04:32:50.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.648 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.648 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.648 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.834200+0000", 2024-04-26T04:32:50.648 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00095841100000000005 2024-04-26T04:32:50.648 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.648 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpRead(3.ds0 45/30 ECSubRead(tid=39, to_read={3:b7d7af40:::smithi17829308-48:head=0,200704,0}, subchunks={3:b7d7af40:::smithi17829308-48:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.700614+0000", 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 10.899664722000001, 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.134188259, 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.700614+0000", 2024-04-26T04:32:50.649 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.700614+0000", 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.700614+0000", 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.650 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.700614+0000", 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.700681+0000", 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7012000000000001e-05 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.800828+0000", 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100146754 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.652 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.834617+0000", 2024-04-26T04:32:50.652 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.033789040999999999 2024-04-26T04:32:50.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.652 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.652 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.834802+0000", 2024-04-26T04:32:50.652 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018545200000000001 2024-04-26T04:32:50.652 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.652 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.652 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.652 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:424 3.ds0 3:b7d7af40:::smithi17829308-48:head [append 0~720896] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:32:50.653 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.921237+0000", 2024-04-26T04:32:50.653 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.6790422429999996, 2024-04-26T04:32:50.653 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40632559499999998, 2024-04-26T04:32:50.653 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.653 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.653 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:50.653 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.653 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.653 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.653 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 424 2024-04-26T04:32:50.653 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.653 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.653 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.921237+0000", 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.921237+0000", 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.921239+0000", 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2900000000000001e-06 2024-04-26T04:32:50.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922033+0000", 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00079434500000000001 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922059+0000", 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5925999999999999e-05 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922072+0000", 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2943e-05 2024-04-26T04:32:50.655 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922191+0000", 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000118581 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922362+0000", 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017091599999999999 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.924002+0000", 2024-04-26T04:32:50.656 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016405359999999999 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.928906+0000", 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0049035720000000001 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327373+0000", 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39846698000000003 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327562+0000", 2024-04-26T04:32:50.657 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018950600000000001 2024-04-26T04:32:50.658 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.658 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.658 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.658 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.658 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.658 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:425 3.ds0 3:b7d7af40:::smithi17829308-48:head [append 0~49152] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:32:50.658 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.922098+0000", 2024-04-26T04:32:50.658 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.6781803990000004, 2024-04-26T04:32:50.658 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.406108466, 2024-04-26T04:32:50.658 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.658 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.658 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:50.658 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.659 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.659 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.659 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 425 2024-04-26T04:32:50.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.659 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.659 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.659 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.659 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922098+0000", 2024-04-26T04:32:50.659 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.659 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.659 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.659 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922098+0000", 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922100+0000", 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2300000000000001e-06 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922189+0000", 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9623999999999994e-05 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.660 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922208+0000", 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8643000000000001e-05 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922219+0000", 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0699999999999999e-05 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.925187+0000", 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0029683930000000002 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.662 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.662 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.925371+0000", 2024-04-26T04:32:50.662 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000184382 2024-04-26T04:32:50.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.662 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.662 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.662 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.926426+0000", 2024-04-26T04:32:50.662 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0010545190000000001 2024-04-26T04:32:50.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.662 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.662 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.662 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.928909+0000", 2024-04-26T04:32:50.662 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.002482981 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328000+0000", 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.399091539 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328207+0000", 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000206455 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:426 3.ds0 3:b7d7af40:::smithi17829308-48:head [setxattr _header (56) in=7b] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.922241+0000", 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.6780382429999996, 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40635279400000002, 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 426 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.664 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.665 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.665 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922241+0000", 2024-04-26T04:32:50.665 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.665 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.665 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.665 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922241+0000", 2024-04-26T04:32:50.665 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.666 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.666 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.666 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922242+0000", 2024-04-26T04:32:50.666 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0470000000000001e-06 2024-04-26T04:32:50.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.666 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.666 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.666 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922246+0000", 2024-04-26T04:32:50.667 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5029999999999999e-06 2024-04-26T04:32:50.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.667 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.667 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922257+0000", 2024-04-26T04:32:50.667 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.059e-05 2024-04-26T04:32:50.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.667 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.667 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922265+0000", 2024-04-26T04:32:50.667 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.8869999999999992e-06 2024-04-26T04:32:50.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.927201+0000", 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0049363640000000004 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.927353+0000", 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015164000000000001 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.928331+0000", 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00097882500000000001 2024-04-26T04:32:50.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.669 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:50.669 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.029157+0000", 2024-04-26T04:32:50.669 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100825282 2024-04-26T04:32:50.669 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.669 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:50.669 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328408+0000", 2024-04-26T04:32:50.669 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29925099700000002 2024-04-26T04:32:50.669 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.669 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.669 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328593+0000", 2024-04-26T04:32:50.670 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018565899999999999 2024-04-26T04:32:50.670 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.670 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.670 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.670 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.670 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:427 3.ds0 3:b7d7af40:::smithi17829308-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:32:50.670 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.922276+0000", 2024-04-26T04:32:50.670 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.6780031360000001, 2024-04-26T04:32:50.670 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.57447843799999998, 2024-04-26T04:32:50.670 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.670 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.670 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:50.671 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:50.671 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:50.671 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:50.671 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 427 2024-04-26T04:32:50.671 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.671 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.671 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.671 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922276+0000", 2024-04-26T04:32:50.671 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.671 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.671 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922276+0000", 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922277+0000", 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.7000000000000003e-07 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922280+0000", 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0589999999999998e-06 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.672 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922293+0000", 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3617e-05 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922302+0000", 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4300000000000006e-06 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.929362+0000", 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.007060439 2024-04-26T04:32:50.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.929534+0000", 2024-04-26T04:32:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017232900000000001 2024-04-26T04:32:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.329547+0000", 2024-04-26T04:32:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40001209799999998 2024-04-26T04:32:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:50.674 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.329786+0000", 2024-04-26T04:32:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00023956800000000001 2024-04-26T04:32:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.496754+0000", 2024-04-26T04:32:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.16696802799999999 2024-04-26T04:32:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 50/30 ECSubWriteReply(tid=47, last_complete=50'30, committed=1, applied=1))", 2024-04-26T04:32:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:45.026315+0000", 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.5739641970000005, 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.200559509, 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.026315+0000", 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.676 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.026315+0000", 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.026322+0000", 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5299999999999999e-06 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.026332+0000", 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.5599999999999999e-06 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.026360+0000", 2024-04-26T04:32:50.677 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8716000000000001e-05 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.126467+0000", 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100106685 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.226847+0000", 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100379626 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.678 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.226874+0000", 2024-04-26T04:32:50.679 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7392000000000001e-05 2024-04-26T04:32:50.679 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.679 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.679 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.679 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.679 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.679 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 50/30 ECSubWriteReply(tid=49, last_complete=50'32, committed=1, applied=1))", 2024-04-26T04:32:50.679 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:45.126605+0000", 2024-04-26T04:32:50.679 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.4736741689999997, 2024-04-26T04:32:50.679 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20061020500000001, 2024-04-26T04:32:50.679 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.679 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.126605+0000", 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.126605+0000", 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.680 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.681 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.126606+0000", 2024-04-26T04:32:50.681 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.47e-07 2024-04-26T04:32:50.681 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.681 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.681 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.681 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.126612+0000", 2024-04-26T04:32:50.681 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.3559999999999998e-06 2024-04-26T04:32:50.681 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.681 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.681 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.681 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.126636+0000", 2024-04-26T04:32:50.681 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4193e-05 2024-04-26T04:32:50.681 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.682 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.682 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.682 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.126652+0000", 2024-04-26T04:32:50.682 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6252999999999998e-05 2024-04-26T04:32:50.682 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.682 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.682 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.682 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327182+0000", 2024-04-26T04:32:50.682 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20052986 2024-04-26T04:32:50.682 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.682 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.682 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.682 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327215+0000", 2024-04-26T04:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2696e-05 2024-04-26T04:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 50/30 ECSubWriteReply(tid=47, last_complete=50'30, committed=1, applied=1))", 2024-04-26T04:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:45.127342+0000", 2024-04-26T04:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.4729368029999996, 2024-04-26T04:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20024288000000001, 2024-04-26T04:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.683 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.684 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.684 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.684 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.684 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127342+0000", 2024-04-26T04:32:50.684 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.684 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.684 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.684 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.684 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127342+0000", 2024-04-26T04:32:50.684 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.684 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.685 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.685 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.685 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127344+0000", 2024-04-26T04:32:50.685 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0269999999999998e-06 2024-04-26T04:32:50.685 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.685 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.685 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.685 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127354+0000", 2024-04-26T04:32:50.685 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.7569999999999999e-06 2024-04-26T04:32:50.685 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.685 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.685 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.685 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127381+0000", 2024-04-26T04:32:50.686 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7622000000000001e-05 2024-04-26T04:32:50.686 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.686 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.686 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.686 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127398+0000", 2024-04-26T04:32:50.686 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.615e-05 2024-04-26T04:32:50.686 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.686 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.686 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.686 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327256+0000", 2024-04-26T04:32:50.686 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.19985855499999999 2024-04-26T04:32:50.686 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.686 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327585+0000", 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00032876900000000002 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 50/30 ECSubWriteReply(tid=48, last_complete=50'31, committed=1, applied=1))", 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:45.127432+0000", 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.4728472909999999, 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20080141000000001, 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.687 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.688 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.688 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.688 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.688 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127432+0000", 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127432+0000", 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127432+0000", 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.0999999999999998e-07 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.689 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.690 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127437+0000", 2024-04-26T04:32:50.690 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.386e-06 2024-04-26T04:32:50.690 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.690 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.690 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.690 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127452+0000", 2024-04-26T04:32:50.690 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5106000000000001e-05 2024-04-26T04:32:50.690 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.690 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.690 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.690 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127461+0000", 2024-04-26T04:32:50.690 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.0829999999999993e-06 2024-04-26T04:32:50.690 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.691 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.691 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.691 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327699+0000", 2024-04-26T04:32:50.691 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.200237834 2024-04-26T04:32:50.691 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.691 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.691 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.691 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328233+0000", 2024-04-26T04:32:50.691 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000534291 2024-04-26T04:32:50.691 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.691 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.691 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.691 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.692 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.692 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 50/30 ECSubWriteReply(tid=49, last_complete=50'32, committed=1, applied=1))", 2024-04-26T04:32:50.692 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:45.127943+0000", 2024-04-26T04:32:50.692 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.472335717, 2024-04-26T04:32:50.692 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20148519600000001, 2024-04-26T04:32:50.692 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.692 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.692 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.692 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.692 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.692 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.692 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127943+0000", 2024-04-26T04:32:50.692 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127943+0000", 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127944+0000", 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3540000000000001e-06 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127953+0000", 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.0360000000000006e-06 2024-04-26T04:32:50.693 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127974+0000", 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1396999999999999e-05 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127986+0000", 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2566e-05 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328290+0000", 2024-04-26T04:32:50.694 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.200303125 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.329428+0000", 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0011387179999999999 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 52/30 ECSubWriteReply(tid=53, last_complete=52'34, committed=1, applied=1))", 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.642501+0000", 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 3.957777911, 2024-04-26T04:32:50.695 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20049150600000001, 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.642501+0000", 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.642501+0000", 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:50.696 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.642502+0000", 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.096e-06 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.642509+0000", 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.3200000000000002e-06 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.642532+0000", 2024-04-26T04:32:50.697 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2629999999999998e-05 2024-04-26T04:32:50.698 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.698 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.698 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:50.698 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.742678+0000", 2024-04-26T04:32:50.698 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100145925 2024-04-26T04:32:50.698 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.698 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.698 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:50.698 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.842940+0000", 2024-04-26T04:32:50.698 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100261742 2024-04-26T04:32:50.698 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:50.698 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:50.698 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:50.699 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.842992+0000", 2024-04-26T04:32:50.699 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.2793000000000001e-05 2024-04-26T04:32:50.699 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.699 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.699 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.699 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:50.699 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:50.699 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:50.701 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T04:32:50.701 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:50.701+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:50.801 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:50.796+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:50.869 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:50.869 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:50.869 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 0 2024-04-26T04:32:50.870 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:50.883 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T04:32:50.902 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:50.901+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:51.003 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:51.001+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:51.005 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:51.005 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:51.005 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:32:51.005 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:32:51.005 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:51.019 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T04:32:51.105 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:51.101+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:51.146 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:51.147 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:32:51.147 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:32:51.147 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:32:51.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.147 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:8 3.fs0 3:f50d0def:::smithi17829308-3:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:51.147 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:32:51.147 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 25.942633873999998, 2024-04-26T04:32:51.147 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51428121599999999, 2024-04-26T04:32:51.147 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.147 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.147 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 8 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.148 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200419+0000", 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9800000000000002e-07 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200433+0000", 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4513e-05 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.149 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200460+0000", 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6438999999999999e-05 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200477+0000", 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7346e-05 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200757+0000", 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000279771 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200766+0000", 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.7159999999999992e-06 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.341113+0000", 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14034733799999999 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.341282+0000", 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000169197 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.342226+0000", 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00094361300000000005 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.343875+0000", 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016493910000000001 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714472+0000", 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.37059684599999998 2024-04-26T04:32:51.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714699+0000", 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022704800000000001 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:9 3.fs0 3:f50d0def:::smithi17829308-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 25.942553185000001, 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71712659899999998, 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.153 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 9 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.154 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1999999999999998e-07 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200506+0000", 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.1399999999999997e-06 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.155 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200520+0000", 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4253000000000001e-05 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200531+0000", 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1375999999999999e-05 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200817+0000", 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00028585699999999998 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.156 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200824+0000", 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6730000000000003e-06 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.342845+0000", 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.142020961 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.343080+0000", 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000235481 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714851+0000", 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.371770347 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.715101+0000", 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00025083100000000002 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917625+0000", 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20252376 2024-04-26T04:32:51.158 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:16 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~98304,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 25.941396183999998, 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.514661445, 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.159 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.160 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 16 2024-04-26T04:32:51.160 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.160 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.160 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.160 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.160 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:32:51.160 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.160 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.160 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.160 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.160 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:32:51.160 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.160 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201657+0000", 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.052e-06 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201822+0000", 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016550000000000001 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201842+0000", 2024-04-26T04:32:51.161 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0231e-05 2024-04-26T04:32:51.162 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.162 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.162 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.162 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201856+0000", 2024-04-26T04:32:51.162 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3234999999999999e-05 2024-04-26T04:32:51.162 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.162 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.162 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.162 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201960+0000", 2024-04-26T04:32:51.162 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0001043 2024-04-26T04:32:51.162 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.162 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.162 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201982+0000", 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1936000000000001e-05 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344166+0000", 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14218376399999999 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344545+0000", 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00037956399999999999 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.163 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.164 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.164 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.346210+0000", 2024-04-26T04:32:51.164 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00166501 2024-04-26T04:32:51.164 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.164 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.164 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.164 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349256+0000", 2024-04-26T04:32:51.164 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0030461080000000001 2024-04-26T04:32:51.164 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.164 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.164 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.164 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716126+0000", 2024-04-26T04:32:51.164 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.366869679 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716317+0000", 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019106599999999999 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:17 3.fs0 3:f03878c8:::smithi17829308-6:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 25.941168093000002, 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51482183299999995, 2024-04-26T04:32:51.165 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.166 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.166 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.166 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.166 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.166 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.166 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 17 2024-04-26T04:32:51.166 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.166 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.166 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.166 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.166 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:32:51.166 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.166 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201885+0000", 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1060000000000001e-06 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201891+0000", 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.806e-06 2024-04-26T04:32:51.167 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.168 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.168 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.168 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201905+0000", 2024-04-26T04:32:51.168 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.432e-05 2024-04-26T04:32:51.168 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.169 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.169 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.169 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201929+0000", 2024-04-26T04:32:51.169 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4417e-05 2024-04-26T04:32:51.169 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.169 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.169 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.169 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202056+0000", 2024-04-26T04:32:51.169 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012684400000000001 2024-04-26T04:32:51.170 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.170 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.170 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:51.170 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202066+0000", 2024-04-26T04:32:51.170 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.5230000000000001e-06 2024-04-26T04:32:51.170 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.170 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.170 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.170 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.347195+0000", 2024-04-26T04:32:51.170 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.145129646 2024-04-26T04:32:51.170 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.170 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.170 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.347408+0000", 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00021322299999999999 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.348249+0000", 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000840363 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.353641+0000", 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0053921280000000004 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.171 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.172 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716520+0000", 2024-04-26T04:32:51.172 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.36287924799999999 2024-04-26T04:32:51.172 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.172 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.172 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.172 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716705+0000", 2024-04-26T04:32:51.172 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018520900000000001 2024-04-26T04:32:51.172 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.172 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.172 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.172 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.172 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.172 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:18 3.fs0 3:f03878c8:::smithi17829308-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 25.941104965000001, 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71589294699999995, 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 18 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.173 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.174 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:32:51.174 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.174 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.174 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.174 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.175 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:32:51.175 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.175 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.175 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.175 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.175 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201948+0000", 2024-04-26T04:32:51.175 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.11e-06 2024-04-26T04:32:51.175 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.175 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.175 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.175 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201962+0000", 2024-04-26T04:32:51.175 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4297999999999999e-05 2024-04-26T04:32:51.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.176 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.176 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.176 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201990+0000", 2024-04-26T04:32:51.176 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8243e-05 2024-04-26T04:32:51.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.176 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.176 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.176 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202004+0000", 2024-04-26T04:32:51.176 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3259e-05 2024-04-26T04:32:51.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.176 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202132+0000", 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000128596 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202149+0000", 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6463000000000001e-05 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349005+0000", 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14685641099999999 2024-04-26T04:32:51.177 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.178 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.178 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:51.178 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349233+0000", 2024-04-26T04:32:51.178 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000227798 2024-04-26T04:32:51.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.178 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.178 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.178 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716805+0000", 2024-04-26T04:32:51.178 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.36757169099999998 2024-04-26T04:32:51.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.178 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.178 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.179 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716944+0000", 2024-04-26T04:32:51.179 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013965200000000001 2024-04-26T04:32:51.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.179 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.179 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.179 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917840+0000", 2024-04-26T04:32:51.179 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20089542599999999 2024-04-26T04:32:51.179 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.179 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.179 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.181 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.181 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:37 3.fs0 3:fa2b8868:::smithi17829308-13:head [append 0~212992,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:51.181 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.308319+0000", 2024-04-26T04:32:51.181 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 25.834732612, 2024-04-26T04:32:51.181 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40929669000000002, 2024-04-26T04:32:51.181 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.182 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.182 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.182 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.182 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.182 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.182 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 37 2024-04-26T04:32:51.182 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.182 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.182 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.182 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.182 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308319+0000", 2024-04-26T04:32:51.182 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.183 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.183 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.183 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.183 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308319+0000", 2024-04-26T04:32:51.183 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.183 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.184 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.184 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.184 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308321+0000", 2024-04-26T04:32:51.184 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.39e-06 2024-04-26T04:32:51.184 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.184 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.185 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.185 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308655+0000", 2024-04-26T04:32:51.185 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00033381099999999999 2024-04-26T04:32:51.185 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.185 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.185 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.185 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308682+0000", 2024-04-26T04:32:51.186 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7112999999999999e-05 2024-04-26T04:32:51.186 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.186 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.186 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.186 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308703+0000", 2024-04-26T04:32:51.186 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0908999999999999e-05 2024-04-26T04:32:51.186 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.186 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.186 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.186 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349359+0000", 2024-04-26T04:32:51.187 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.040655934999999997 2024-04-26T04:32:51.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.187 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.187 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.187 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349607+0000", 2024-04-26T04:32:51.187 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00024798700000000002 2024-04-26T04:32:51.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.187 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.187 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.188 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.350900+0000", 2024-04-26T04:32:51.188 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0012927940000000001 2024-04-26T04:32:51.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.188 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.188 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.188 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.353649+0000", 2024-04-26T04:32:51.188 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0027486519999999999 2024-04-26T04:32:51.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.188 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.188 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.188 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.717506+0000", 2024-04-26T04:32:51.188 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.36385687700000002 2024-04-26T04:32:51.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.189 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.189 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.189 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.717616+0000", 2024-04-26T04:32:51.189 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000110222 2024-04-26T04:32:51.189 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.189 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.190 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.190 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.190 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:38 3.fs0 3:fa2b8868:::smithi17829308-13:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:51.190 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.308739+0000", 2024-04-26T04:32:51.190 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 25.834313093999999, 2024-04-26T04:32:51.190 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50622445199999999, 2024-04-26T04:32:51.191 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.191 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.191 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.191 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.191 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.191 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.191 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 38 2024-04-26T04:32:51.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.192 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.192 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.192 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.192 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308739+0000", 2024-04-26T04:32:51.192 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.192 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.193 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.193 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308739+0000", 2024-04-26T04:32:51.193 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.193 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.193 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.194 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.194 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308740+0000", 2024-04-26T04:32:51.194 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7570000000000001e-06 2024-04-26T04:32:51.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.194 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.194 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.194 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308749+0000", 2024-04-26T04:32:51.195 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.833e-06 2024-04-26T04:32:51.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.195 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.195 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.195 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308772+0000", 2024-04-26T04:32:51.195 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2779e-05 2024-04-26T04:32:51.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.196 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.196 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.196 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308787+0000", 2024-04-26T04:32:51.196 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5532999999999998e-05 2024-04-26T04:32:51.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.196 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.196 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.196 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.351662+0000", 2024-04-26T04:32:51.197 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.042874774999999997 2024-04-26T04:32:51.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.197 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.197 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.197 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.351936+0000", 2024-04-26T04:32:51.197 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00027371399999999999 2024-04-26T04:32:51.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.198 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.198 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.198 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.353132+0000", 2024-04-26T04:32:51.198 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0011959539999999999 2024-04-26T04:32:51.198 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.199 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.199 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.199 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.354065+0000", 2024-04-26T04:32:51.199 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000933343 2024-04-26T04:32:51.199 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.199 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.200 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.200 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.814887+0000", 2024-04-26T04:32:51.200 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.46082132199999998 2024-04-26T04:32:51.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.200 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.200 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.200 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.814963+0000", 2024-04-26T04:32:51.201 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.6441999999999998e-05 2024-04-26T04:32:51.201 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.201 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.201 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.201 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.201 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.201 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:243 3.fs0 3:fa2b8868:::smithi17829308-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:51.201 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.767317+0000", 2024-04-26T04:32:51.201 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 21.375734945000001, 2024-04-26T04:32:51.201 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.307000616, 2024-04-26T04:32:51.201 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.201 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.202 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:51.202 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.202 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.202 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.202 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 243 2024-04-26T04:32:51.202 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.202 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.202 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.202 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.202 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767317+0000", 2024-04-26T04:32:51.202 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.202 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.203 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.203 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.203 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767317+0000", 2024-04-26T04:32:51.203 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.203 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.203 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.203 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.203 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767319+0000", 2024-04-26T04:32:51.203 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.81e-06 2024-04-26T04:32:51.203 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767326+0000", 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5499999999999997e-06 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767369+0000", 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3291999999999998e-05 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.767379+0000", 2024-04-26T04:32:51.204 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.2560000000000008e-06 2024-04-26T04:32:51.205 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.205 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.205 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.205 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.869882+0000", 2024-04-26T04:32:51.205 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10250316 2024-04-26T04:32:51.205 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.205 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.205 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.205 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.870039+0000", 2024-04-26T04:32:51.205 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015685499999999999 2024-04-26T04:32:51.205 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.205 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.206 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.206 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.074317+0000", 2024-04-26T04:32:51.206 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20427869300000001 2024-04-26T04:32:51.206 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.206 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.206 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.206 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.206 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.7s6 40/30 ECSubWrite(tid=111, reqid=client.4670.0:335, at_version=40'11, trim_to=0'0, roll_forward_to=36'10))", 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.145572+0000", 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.997480136, 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30427213800000003, 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.145572+0000", 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.207 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.208 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.145572+0000", 2024-04-26T04:32:51.208 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.208 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.208 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.208 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.208 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.145573+0000", 2024-04-26T04:32:51.208 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.593e-06 2024-04-26T04:32:51.208 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.208 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.208 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.209 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.146119+0000", 2024-04-26T04:32:51.209 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00054548300000000004 2024-04-26T04:32:51.209 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.209 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.209 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.209 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.146138+0000", 2024-04-26T04:32:51.209 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9196000000000001e-05 2024-04-26T04:32:51.209 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.209 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.209 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.209 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.146148+0000", 2024-04-26T04:32:51.210 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8980000000000008e-06 2024-04-26T04:32:51.210 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.210 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.210 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.210 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.246392+0000", 2024-04-26T04:32:51.210 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100244703 2024-04-26T04:32:51.210 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.210 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.210 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.210 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.246434+0000", 2024-04-26T04:32:51.210 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.1742000000000001e-05 2024-04-26T04:32:51.210 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.210 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.211 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.211 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.449785+0000", 2024-04-26T04:32:51.211 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20335057100000001 2024-04-26T04:32:51.211 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.211 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.211 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.211 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.449844+0000", 2024-04-26T04:32:51.211 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8952000000000001e-05 2024-04-26T04:32:51.211 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.211 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.211 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.211 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.211 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.212 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:346 3.fs0 3:fa2b8868:::smithi17829308-13:head [copy-from ver 14] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:51.212 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.181230+0000", 2024-04-26T04:32:51.212 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.961822185000001, 2024-04-26T04:32:51.212 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.80960196600000001, 2024-04-26T04:32:51.212 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.212 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.212 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.212 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.212 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 346 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181230+0000", 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181230+0000", 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.213 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181232+0000", 2024-04-26T04:32:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3609999999999999e-06 2024-04-26T04:32:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181257+0000", 2024-04-26T04:32:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5100999999999999e-05 2024-04-26T04:32:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.214 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181283+0000", 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5612000000000001e-05 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181342+0000", 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9744000000000003e-05 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248343+0000", 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.067000834999999995 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.215 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248487+0000", 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014416900000000001 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.787056+0000", 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.53856828300000004 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.789372+0000", 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0023167069999999999 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.216 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.217 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.217 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.990698+0000", 2024-04-26T04:32:51.217 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201325688 2024-04-26T04:32:51.217 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.217 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.217 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.217 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.990831+0000", 2024-04-26T04:32:51.217 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000133466 2024-04-26T04:32:51.217 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.217 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.217 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.217 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.217 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:347 3.fs0 3:fa2b8868:::smithi17829308-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.181534+0000", 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.961517626999999, 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.60738179000000003, 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 347 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.218 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181534+0000", 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181534+0000", 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181535+0000", 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0640000000000001e-06 2024-04-26T04:32:51.219 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181541+0000", 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4099999999999999e-06 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181557+0000", 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6728999999999999e-05 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181570+0000", 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2743e-05 2024-04-26T04:32:51.220 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248864+0000", 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.067293511 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248995+0000", 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013142000000000001 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.788613+0000", 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.53961845799999997 2024-04-26T04:32:51.221 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.788760+0000", 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014664600000000001 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.788916+0000", 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000155809 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.222 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(osd.3.22:1 3.fs0 3:f8001b32:::smithi17829308-38:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.221684+0000", 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.921367671, 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.353815977, 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221684+0000", 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.223 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221684+0000", 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221685+0000", 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4410000000000001e-06 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221692+0000", 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.1099999999999999e-06 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.224 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221703+0000", 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.111e-05 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221714+0000", 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1473e-05 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.448870+0000", 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.227155934 2024-04-26T04:32:51.225 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.448943+0000", 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.2631000000000003e-05 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.575500+0000", 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.126557278 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.226 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:430 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~622592] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.182041+0000", 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.9610106580000002, 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51141810899999995, 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 430 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.227 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182041+0000", 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182041+0000", 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182044+0000", 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.83e-06 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.228 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182758+0000", 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00071432400000000001 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182792+0000", 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3655999999999998e-05 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.282928+0000", 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100136005 2024-04-26T04:32:51.229 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.283058+0000", 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013049700000000001 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.283071+0000", 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3106e-05 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.326367+0000", 2024-04-26T04:32:51.230 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.043295412999999998 2024-04-26T04:32:51.231 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.231 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.231 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.231 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.326694+0000", 2024-04-26T04:32:51.231 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00032741600000000002 2024-04-26T04:32:51.231 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.231 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.231 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.231 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.329727+0000", 2024-04-26T04:32:51.231 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0030326509999999999 2024-04-26T04:32:51.231 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.231 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.333822+0000", 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0040948139999999996 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.693241+0000", 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.35941913199999997 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.693459+0000", 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00021826500000000001 2024-04-26T04:32:51.232 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.233 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.233 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.233 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.233 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.233 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:431 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~475136] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:32:51.233 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.282983+0000", 2024-04-26T04:32:51.233 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.8600682040000001, 2024-04-26T04:32:51.233 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.411303902, 2024-04-26T04:32:51.233 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.233 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.233 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.233 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.234 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.234 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.234 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 431 2024-04-26T04:32:51.234 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.234 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.234 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.234 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.234 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.282983+0000", 2024-04-26T04:32:51.234 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.234 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.234 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.234 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.234 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.282983+0000", 2024-04-26T04:32:51.235 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.235 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.235 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.235 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.235 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.282985+0000", 2024-04-26T04:32:51.235 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.933e-06 2024-04-26T04:32:51.235 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.235 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.235 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.235 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.283498+0000", 2024-04-26T04:32:51.235 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00051254800000000004 2024-04-26T04:32:51.235 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.235 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.236 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.236 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.283522+0000", 2024-04-26T04:32:51.236 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4049e-05 2024-04-26T04:32:51.236 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.236 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.236 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.236 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383665+0000", 2024-04-26T04:32:51.236 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100142975 2024-04-26T04:32:51.236 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.236 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.236 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.236 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383825+0000", 2024-04-26T04:32:51.236 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015984699999999999 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384053+0000", 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022778800000000001 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.385649+0000", 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001596164 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.237 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.592570+0000", 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20692076200000001 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.694095+0000", 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10152514 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.694287+0000", 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019269600000000001 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.238 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:432 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~294912] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.383720+0000", 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.7593313769999996, 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.352655626, 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 432 2024-04-26T04:32:51.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383720+0000", 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383720+0000", 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383723+0000", 2024-04-26T04:32:51.240 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7839999999999999e-06 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383953+0000", 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00023033799999999999 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383985+0000", 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1751999999999997e-05 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.241 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384006+0000", 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.05e-05 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.386759+0000", 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0027536240000000001 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.387014+0000", 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000255002 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.242 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.243 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.388836+0000", 2024-04-26T04:32:51.243 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001821528 2024-04-26T04:32:51.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.243 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.243 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.592573+0000", 2024-04-26T04:32:51.243 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.203737154 2024-04-26T04:32:51.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.243 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.243 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.243 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.736257+0000", 2024-04-26T04:32:51.243 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14368439599999999 2024-04-26T04:32:51.243 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.736376+0000", 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000118548 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:433 3.fs0 3:f03878c8:::smithi17829308-6:head [setxattr _header (55) in=7b] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.384045+0000", 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.7590070669999998, 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.41074539900000001, 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.244 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 433 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384045+0000", 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.245 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384045+0000", 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384047+0000", 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1100000000000001e-06 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384056+0000", 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.0100000000000001e-06 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.246 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.247 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.247 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384077+0000", 2024-04-26T04:32:51.247 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1546e-05 2024-04-26T04:32:51.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.247 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.247 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384094+0000", 2024-04-26T04:32:51.247 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6305999999999999e-05 2024-04-26T04:32:51.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.247 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.247 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.247 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.390024+0000", 2024-04-26T04:32:51.247 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0059306819999999996 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.390289+0000", 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026486100000000002 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.391188+0000", 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00089867300000000001 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.592830+0000", 2024-04-26T04:32:51.248 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20164169200000001 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.794632+0000", 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201802486 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.794790+0000", 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000158033 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.250 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.250 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:434 3.fs0 3:f03878c8:::smithi17829308-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:32:51.250 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.384120+0000", 2024-04-26T04:32:51.250 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.7589318089999999, 2024-04-26T04:32:51.250 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81328199499999998, 2024-04-26T04:32:51.250 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.250 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:51.250 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.250 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 434 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384120+0000", 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384120+0000", 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384121+0000", 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3069999999999999e-06 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384127+0000", 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4430000000000002e-06 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384147+0000", 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0533000000000001e-05 2024-04-26T04:32:51.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384163+0000", 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5948999999999999e-05 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.491986+0000", 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.107822418 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.492246+0000", 2024-04-26T04:32:51.253 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026025100000000003 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.796841+0000", 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30459499299999998 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.796996+0000", 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015495 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:47.197402+0000", 2024-04-26T04:32:51.254 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40040615099999999 2024-04-26T04:32:51.255 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.255 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.255 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.255 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.255 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.255 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:469 3.fs0 3:f0b18d84:::smithi17829308-36:head [read 0~589824,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:32:51.255 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.185844+0000", 2024-04-26T04:32:51.255 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.95720777700000004, 2024-04-26T04:32:51.255 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30605939399999998, 2024-04-26T04:32:51.255 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.258 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.258 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:51.258 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.258 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.258 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.258 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 469 2024-04-26T04:32:51.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.258 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.258 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.258 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.258 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185844+0000", 2024-04-26T04:32:51.258 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185844+0000", 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185848+0000", 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.8940000000000003e-06 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185864+0000", 2024-04-26T04:32:51.259 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6024000000000001e-05 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185891+0000", 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6886e-05 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185905+0000", 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4617000000000001e-05 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.186003+0000", 2024-04-26T04:32:51.260 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8090000000000004e-05 2024-04-26T04:32:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:32:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.186013+0000", 2024-04-26T04:32:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8050000000000001e-06 2024-04-26T04:32:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.287440+0000", 2024-04-26T04:32:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10142660100000001 2024-04-26T04:32:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.261 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.287722+0000", 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000282501 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.491903+0000", 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20418097599999999 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:470 3.fs0 3:f0b18d84:::smithi17829308-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:32:51.262 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.185938+0000", 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.957113726, 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.306666406, 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 470 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185938+0000", 2024-04-26T04:32:51.263 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185938+0000", 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185939+0000", 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.7199999999999999e-07 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.264 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185945+0000", 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.737e-06 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185952+0000", 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7519999999999999e-06 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185958+0000", 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8499999999999999e-06 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.265 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.186105+0000", 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014695499999999999 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.186112+0000", 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5469999999999999e-06 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.288839+0000", 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.102726944 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.266 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.288974+0000", 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013509700000000001 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.492604+0000", 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.203629952 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:471 3.fs0 3:f0b18d84:::smithi17829308-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:32:51.267 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.185982+0000", 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.95706999000000004, 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40798264899999998, 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 471 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.268 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185982+0000", 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185982+0000", 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185982+0000", 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.1300000000000001e-07 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.269 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185985+0000", 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6309999999999999e-06 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185991+0000", 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8409999999999996e-06 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185995+0000", 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5689999999999998e-06 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.270 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.286274+0000", 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10027858000000001 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.286295+0000", 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1140000000000001e-05 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.390661+0000", 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10436590699999999 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.271 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.390758+0000", 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.7398999999999994e-05 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.593964+0000", 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20320616899999999 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.272 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:51.275 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.13 to 1 (10000) 2024-04-26T04:32:51.275 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T04:32:51.276 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-04-26T04:32:51.276 INFO:tasks.rados.rados.0.smithi178.stdout:190: done (3 left) 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 2 current snap is 12 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:191: seq_num 88 ranges {3670016=720896,4390912=442368,4833280=327680} 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:191: writing smithi17829308-2 from 3670016 to 4390912 tid 1 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:191: writing smithi17829308-2 from 4390912 to 4833280 tid 2 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:191: writing smithi17829308-2 from 4833280 to 5160960 tid 3 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:192: copy_from oid 38 from oid 46 current snap is 12 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:188: finishing write tid 1 to smithi17829308-9 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:187: finishing write tid 1 to smithi17829308-1 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:187: finishing write tid 2 to smithi17829308-1 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:187: finishing write tid 3 to smithi17829308-1 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:186: expect (ObjNum 35 snap 0 seq_num 35) 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:187: finishing write tid 4 to smithi17829308-1 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:187: finishing write tid 5 to smithi17829308-1 2024-04-26T04:32:51.277 INFO:tasks.rados.rados.0.smithi178.stdout:187: oid 1 updating version 0 to 60 2024-04-26T04:32:51.278 INFO:tasks.rados.rados.0.smithi178.stdout:187: oid 1 version 60 is already newer than 57 2024-04-26T04:32:51.278 INFO:tasks.rados.rados.0.smithi178.stdout:187: oid 1 version 60 is already newer than 59 2024-04-26T04:32:51.278 INFO:tasks.rados.rados.0.smithi178.stdout:187: oid 1 version 60 is already newer than 58 2024-04-26T04:32:51.278 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 1 v 60 (ObjNum 86 snap 12 seq_num 86) dirty exists 2024-04-26T04:32:51.278 INFO:tasks.rados.rados.0.smithi178.stdout:187: left oid 1 (ObjNum 86 snap 12 seq_num 86) 2024-04-26T04:32:51.278 INFO:tasks.rados.rados.0.smithi178.stdout:188: finishing write tid 2 to smithi17829308-9 2024-04-26T04:32:51.278 INFO:tasks.rados.rados.0.smithi178.stdout:188: finishing write tid 3 to smithi17829308-9 2024-04-26T04:32:51.278 INFO:tasks.rados.rados.0.smithi178.stdout:188: finishing write tid 4 to smithi17829308-9 2024-04-26T04:32:51.278 INFO:tasks.rados.rados.0.smithi178.stdout:188: finishing write tid 5 to smithi17829308-9 2024-04-26T04:32:51.278 INFO:tasks.rados.rados.0.smithi178.stdout:188: oid 9 updating version 0 to 42 2024-04-26T04:32:51.279 INFO:tasks.rados.rados.0.smithi178.stdout:188: oid 9 version 42 is already newer than 41 2024-04-26T04:32:51.279 INFO:tasks.rados.rados.0.smithi178.stdout:188: oid 9 updating version 42 to 43 2024-04-26T04:32:51.279 INFO:tasks.rados.rados.0.smithi178.stdout:188: oid 9 updating version 43 to 44 2024-04-26T04:32:51.279 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 9 v 44 (ObjNum 87 snap 12 seq_num 87) dirty exists 2024-04-26T04:32:51.279 INFO:tasks.rados.rados.0.smithi178.stdout:188: left oid 9 (ObjNum 87 snap 12 seq_num 87) 2024-04-26T04:32:51.279 INFO:tasks.rados.rados.0.smithi178.stdout:186: done (4 left) 2024-04-26T04:32:51.279 INFO:tasks.rados.rados.0.smithi178.stdout:187: done (3 left) 2024-04-26T04:32:51.279 INFO:tasks.rados.rados.0.smithi178.stdout:188: done (2 left) 2024-04-26T04:32:51.279 INFO:tasks.rados.rados.0.smithi178.stdout:193: snap_remove snap 8 2024-04-26T04:32:51.279 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:51.270+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:51.374 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:51.373+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:51.446 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:51.447 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:51.447 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 0 2024-04-26T04:32:51.447 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:51.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T04:32:51.511 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:51.510+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:51.576 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:51.573+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:51.589 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:51.589 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:51.590 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:32:51.590 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:32:51.590 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:51.603 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T04:32:51.677 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:51.677+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:51.776 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:19 3.5s0 3:a4a2788c:::smithi17829308-7:head [append 0~114688,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.204107+0000", 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 26.568934012, 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40419384899999999, 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.777 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 19 2024-04-26T04:32:51.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.778 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.778 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.778 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.204107+0000", 2024-04-26T04:32:51.778 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.778 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.778 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.204107+0000", 2024-04-26T04:32:51.778 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.779 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.779 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.204111+0000", 2024-04-26T04:32:51.779 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.6399999999999999e-06 2024-04-26T04:32:51.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.779 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.779 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.204339+0000", 2024-04-26T04:32:51.779 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022846499999999999 2024-04-26T04:32:51.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.779 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.779 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.204382+0000", 2024-04-26T04:32:51.779 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.2667999999999999e-05 2024-04-26T04:32:51.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.780 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.780 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304483+0000", 2024-04-26T04:32:51.780 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100100909 2024-04-26T04:32:51.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.780 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.780 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.404672+0000", 2024-04-26T04:32:51.780 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100189713 2024-04-26T04:32:51.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.780 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.781 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.404819+0000", 2024-04-26T04:32:51.781 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014648800000000001 2024-04-26T04:32:51.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.781 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.781 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.781 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.405632+0000", 2024-04-26T04:32:51.781 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00081290099999999999 2024-04-26T04:32:51.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.781 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.781 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.782 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507692+0000", 2024-04-26T04:32:51.782 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.102059917 2024-04-26T04:32:51.782 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.782 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.782 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608229+0000", 2024-04-26T04:32:51.782 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100537168 2024-04-26T04:32:51.782 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.782 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.782 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608301+0000", 2024-04-26T04:32:51.782 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1979999999999999e-05 2024-04-26T04:32:51.782 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.783 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.783 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.783 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.783 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.783 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:20 3.5s0 3:a4a2788c:::smithi17829308-7:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:51.783 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304525+0000", 2024-04-26T04:32:51.783 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 26.468515822000001, 2024-04-26T04:32:51.783 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.41153016100000001, 2024-04-26T04:32:51.783 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.783 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.783 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.783 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.783 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.784 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.784 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 20 2024-04-26T04:32:51.784 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.784 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.784 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.784 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.784 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304525+0000", 2024-04-26T04:32:51.784 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.784 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.784 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.784 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.784 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304525+0000", 2024-04-26T04:32:51.784 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304529+0000", 2024-04-26T04:32:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2559999999999998e-06 2024-04-26T04:32:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304561+0000", 2024-04-26T04:32:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2398999999999999e-05 2024-04-26T04:32:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.786 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304583+0000", 2024-04-26T04:32:51.786 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2079e-05 2024-04-26T04:32:51.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.786 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.786 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304600+0000", 2024-04-26T04:32:51.786 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6626999999999999e-05 2024-04-26T04:32:51.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.786 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.786 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.406379+0000", 2024-04-26T04:32:51.786 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101779781 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.406585+0000", 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000205985 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.407169+0000", 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00058321299999999996 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.787 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507899+0000", 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100730483 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.715960+0000", 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20806137199999999 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716055+0000", 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.4965999999999998e-05 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.788 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:21 3.5s0 3:a4a2788c:::smithi17829308-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 26.468418092, 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71502945500000004, 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 21 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.789 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.790 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.790 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.790 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:32:51.790 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.790 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.790 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.790 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.790 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:32:51.790 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.790 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.790 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.791 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.791 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304624+0000", 2024-04-26T04:32:51.791 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.186e-06 2024-04-26T04:32:51.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.791 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.791 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304629+0000", 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3630000000000004e-06 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304643+0000", 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4800000000000001e-05 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304654+0000", 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0665999999999999e-05 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.793 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.793 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507573+0000", 2024-04-26T04:32:51.793 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20291931099999999 2024-04-26T04:32:51.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.793 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:51.793 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507644+0000", 2024-04-26T04:32:51.793 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1131999999999996e-05 2024-04-26T04:32:51.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.793 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.793 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.816490+0000", 2024-04-26T04:32:51.793 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30884542700000001 2024-04-26T04:32:51.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.794 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.794 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.816596+0000", 2024-04-26T04:32:51.794 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000105678 2024-04-26T04:32:51.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.794 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.794 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.019652+0000", 2024-04-26T04:32:51.794 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20305689199999999 2024-04-26T04:32:51.794 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.794 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.795 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.795 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:25 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~147456,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:51.795 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:32:51.795 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 26.468293364000001, 2024-04-26T04:32:51.795 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61210120899999998, 2024-04-26T04:32:51.795 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.795 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.795 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.795 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.795 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.795 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 25 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.796 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304749+0000", 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.277e-06 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304925+0000", 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017555399999999999 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304943+0000", 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8529000000000002e-05 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.797 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304954+0000", 2024-04-26T04:32:51.798 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0879999999999999e-05 2024-04-26T04:32:51.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.798 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.798 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.798 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608582+0000", 2024-04-26T04:32:51.798 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30362831899999998 2024-04-26T04:32:51.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.798 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.798 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.798 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608711+0000", 2024-04-26T04:32:51.798 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012887 2024-04-26T04:32:51.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609284+0000", 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00057331000000000005 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.710054+0000", 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100769761 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.916732+0000", 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20667817799999999 2024-04-26T04:32:51.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.800 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.800 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.800 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.916849+0000", 2024-04-26T04:32:51.800 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000116531 2024-04-26T04:32:51.800 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.800 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.800 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.800 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.800 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:26 3.5s0 3:afb9923e:::smithi17829308-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:51.800 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:32:51.800 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 26.468071561999999, 2024-04-26T04:32:51.800 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71271566600000003, 2024-04-26T04:32:51.801 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.801 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.801 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.801 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.801 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.801 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.801 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 26 2024-04-26T04:32:51.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.801 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.802 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.802 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:32:51.802 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.802 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.802 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.802 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:32:51.802 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.802 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.802 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.802 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:32:51.802 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.2399999999999996e-07 2024-04-26T04:32:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304975+0000", 2024-04-26T04:32:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.6460000000000001e-06 2024-04-26T04:32:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304984+0000", 2024-04-26T04:32:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1540000000000008e-06 2024-04-26T04:32:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304992+0000", 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5259999999999996e-06 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609850+0000", 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.304857769 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609963+0000", 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011346899999999999 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.804 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.610506+0000", 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00054322400000000003 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.710058+0000", 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.099551985999999995 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017545+0000", 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30748642599999998 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.805 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.806 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017685+0000", 2024-04-26T04:32:51.806 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014054200000000001 2024-04-26T04:32:51.806 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.806 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.806 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.806 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.806 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:266 3.5s0 3:a5af5236:::smithi17829308-45:head [stat,delete] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:51.806 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.817699+0000", 2024-04-26T04:32:51.806 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 21.955342007999999, 2024-04-26T04:32:51.806 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30440724800000002, 2024-04-26T04:32:51.806 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.806 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.807 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.807 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.807 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.807 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.807 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 266 2024-04-26T04:32:51.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.807 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.807 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.807 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817699+0000", 2024-04-26T04:32:51.807 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.808 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.808 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817699+0000", 2024-04-26T04:32:51.808 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.808 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.808 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.808 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817702+0000", 2024-04-26T04:32:51.808 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.4029999999999999e-06 2024-04-26T04:32:51.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.808 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.808 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.808 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817718+0000", 2024-04-26T04:32:51.808 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.558e-05 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817754+0000", 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.5908999999999998e-05 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817778+0000", 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4221999999999999e-05 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.809 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817892+0000", 2024-04-26T04:32:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000114041 2024-04-26T04:32:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.818024+0000", 2024-04-26T04:32:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013207099999999999 2024-04-26T04:32:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.818890+0000", 2024-04-26T04:32:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00086597099999999999 2024-04-26T04:32:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.810 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.820245+0000", 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001354856 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.121917+0000", 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30167228699999998 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.122106+0000", 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000188908 2024-04-26T04:32:51.811 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.812 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.812 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.812 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.812 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.6s3 40/30 ECSubWrite(tid=16, reqid=client.4670.0:339, at_version=40'9, trim_to=0'0, roll_forward_to=40'7))", 2024-04-26T04:32:51.812 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.172358+0000", 2024-04-26T04:32:51.812 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.600683435000001, 2024-04-26T04:32:51.812 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20383611200000001, 2024-04-26T04:32:51.812 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.813 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.813 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:51.813 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.813 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.813 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.172358+0000", 2024-04-26T04:32:51.813 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.813 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.813 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.172358+0000", 2024-04-26T04:32:51.813 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.814 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.814 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.814 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.172360+0000", 2024-04-26T04:32:51.814 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.063e-06 2024-04-26T04:32:51.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.814 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.814 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.814 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.172708+0000", 2024-04-26T04:32:51.814 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000348127 2024-04-26T04:32:51.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.814 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.814 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.814 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.172748+0000", 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.0546000000000001e-05 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.272889+0000", 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10014091999999999 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.273042+0000", 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000152514 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.815 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.273057+0000", 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5580999999999999e-05 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.376167+0000", 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103109149 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.376194+0000", 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7212e-05 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.816 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:359 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.340615+0000", 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.432426501999998, 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.44310049899999998, 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 359 2024-04-26T04:32:51.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340615+0000", 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340615+0000", 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.818 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340617+0000", 2024-04-26T04:32:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9040000000000001e-06 2024-04-26T04:32:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340632+0000", 2024-04-26T04:32:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.451e-05 2024-04-26T04:32:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340661+0000", 2024-04-26T04:32:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8632e-05 2024-04-26T04:32:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.819 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340676+0000", 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5654e-05 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340769+0000", 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.2292000000000003e-05 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340777+0000", 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.1629999999999996e-06 2024-04-26T04:32:51.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.475834+0000", 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.13505704499999999 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.475929+0000", 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.5441999999999999e-05 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.476694+0000", 2024-04-26T04:32:51.821 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00076508399999999997 2024-04-26T04:32:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.682599+0000", 2024-04-26T04:32:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20590514700000001 2024-04-26T04:32:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.822 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.783507+0000", 2024-04-26T04:32:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100907662 2024-04-26T04:32:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.783715+0000", 2024-04-26T04:32:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00020796399999999999 2024-04-26T04:32:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:360 3.5s0 3:a44e1dec:::smithi17829308-26:head [rollback 2] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:32:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.340709+0000", 2024-04-26T04:32:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.432332138, 2024-04-26T04:32:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.54345505299999997, 2024-04-26T04:32:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.824 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 360 2024-04-26T04:32:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340709+0000", 2024-04-26T04:32:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340709+0000", 2024-04-26T04:32:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.825 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340710+0000", 2024-04-26T04:32:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9100000000000003e-07 2024-04-26T04:32:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340714+0000", 2024-04-26T04:32:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.1760000000000003e-06 2024-04-26T04:32:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340725+0000", 2024-04-26T04:32:51.826 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1664e-05 2024-04-26T04:32:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340734+0000", 2024-04-26T04:32:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8899999999999996e-06 2024-04-26T04:32:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340844+0000", 2024-04-26T04:32:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000110005 2024-04-26T04:32:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.827 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340851+0000", 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7830000000000001e-06 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.577334+0000", 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23648275699999999 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.577565+0000", 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00023120000000000001 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.828 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579037+0000", 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00147218 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.682611+0000", 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103574176 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.884048+0000", 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201436371 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.884164+0000", 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011616 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:361 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.340770+0000", 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.432270918, 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.644132284, 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.830 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 361 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340770+0000", 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.831 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340770+0000", 2024-04-26T04:32:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340771+0000", 2024-04-26T04:32:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8400000000000004e-07 2024-04-26T04:32:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.832 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340774+0000", 2024-04-26T04:32:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.349e-06 2024-04-26T04:32:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.833 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340783+0000", 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.3470000000000005e-06 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340792+0000", 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8740000000000001e-06 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340874+0000", 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.1458000000000005e-05 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.834 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340879+0000", 2024-04-26T04:32:51.835 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4110000000000004e-06 2024-04-26T04:32:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579882+0000", 2024-04-26T04:32:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23900315899999999 2024-04-26T04:32:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.836 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.580145+0000", 2024-04-26T04:32:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026314300000000002 2024-04-26T04:32:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.581275+0000", 2024-04-26T04:32:51.837 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001129997 2024-04-26T04:32:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.682889+0000", 2024-04-26T04:32:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101613946 2024-04-26T04:32:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.984790+0000", 2024-04-26T04:32:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30190081600000002 2024-04-26T04:32:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.838 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.984902+0000", 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0001122 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(osd.11.22:5 3.5s0 3:ac36f45f:::smithi17829308-24:head [assert-version v14,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.374016+0000", 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.399025041000002, 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40911971699999999, 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.839 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.374016+0000", 2024-04-26T04:32:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.840 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.374016+0000", 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.374017+0000", 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3170000000000001e-06 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.374022+0000", 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.729e-06 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.841 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.374032+0000", 2024-04-26T04:32:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9080000000000007e-06 2024-04-26T04:32:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.374040+0000", 2024-04-26T04:32:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.1510000000000004e-06 2024-04-26T04:32:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.842 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.374125+0000", 2024-04-26T04:32:51.843 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4454000000000002e-05 2024-04-26T04:32:51.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.844 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.844 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.844 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.374209+0000", 2024-04-26T04:32:51.844 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4109000000000004e-05 2024-04-26T04:32:51.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.844 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.844 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.844 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.783136+0000", 2024-04-26T04:32:51.844 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40892704899999999 2024-04-26T04:32:51.844 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.844 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.844 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.845 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.845 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.6s3 52/30 ECSubWrite(tid=28, reqid=client.4670.0:440, at_version=52'18, trim_to=0'0, roll_forward_to=44'16))", 2024-04-26T04:32:51.845 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.333639+0000", 2024-04-26T04:32:51.845 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.4394020479999998, 2024-04-26T04:32:51.845 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20842344900000001, 2024-04-26T04:32:51.845 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.845 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.845 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:51.845 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.846 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.846 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.846 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.333639+0000", 2024-04-26T04:32:51.846 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.846 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.333639+0000", 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.333640+0000", 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.3699999999999999e-07 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.333858+0000", 2024-04-26T04:32:51.847 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00021857800000000001 2024-04-26T04:32:51.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.333894+0000", 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.5113000000000001e-05 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.333919+0000", 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5432999999999999e-05 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.537621+0000", 2024-04-26T04:32:51.849 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.203702302 2024-04-26T04:32:51.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.850 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.850 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.850 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.537649+0000", 2024-04-26T04:32:51.850 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7438999999999999e-05 2024-04-26T04:32:51.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.850 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.850 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.850 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.541995+0000", 2024-04-26T04:32:51.850 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0043465980000000001 2024-04-26T04:32:51.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.851 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.851 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.851 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.542062+0000", 2024-04-26T04:32:51.851 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7149000000000004e-05 2024-04-26T04:32:51.851 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.851 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.851 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.851 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.851 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.851 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:477 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~704512] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:32:51.851 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.210833+0000", 2024-04-26T04:32:51.851 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 1.5622081479999999, 2024-04-26T04:32:51.851 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21660977200000001, 2024-04-26T04:32:51.852 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.852 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.852 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.852 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.852 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.852 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.852 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 477 2024-04-26T04:32:51.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.852 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.852 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.852 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.853 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.210833+0000", 2024-04-26T04:32:51.853 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.853 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.853 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.853 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.853 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.210833+0000", 2024-04-26T04:32:51.853 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.853 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.853 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.853 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.854 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.210836+0000", 2024-04-26T04:32:51.854 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1329999999999999e-06 2024-04-26T04:32:51.854 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.855 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.855 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.211282+0000", 2024-04-26T04:32:51.855 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00044637600000000001 2024-04-26T04:32:51.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.855 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.855 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.211311+0000", 2024-04-26T04:32:51.855 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8564e-05 2024-04-26T04:32:51.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.855 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.211327+0000", 2024-04-26T04:32:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6209e-05 2024-04-26T04:32:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.211432+0000", 2024-04-26T04:32:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010431 2024-04-26T04:32:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:32:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.211440+0000", 2024-04-26T04:32:51.856 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.6419999999999996e-06 2024-04-26T04:32:51.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.857 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.857 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.215102+0000", 2024-04-26T04:32:51.857 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0036619180000000001 2024-04-26T04:32:51.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.857 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.857 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.215289+0000", 2024-04-26T04:32:51.857 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018647600000000001 2024-04-26T04:32:51.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.858 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.858 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.858 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.217645+0000", 2024-04-26T04:32:51.858 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0023568590000000002 2024-04-26T04:32:51.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.858 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.858 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.858 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321487+0000", 2024-04-26T04:32:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10384199 2024-04-26T04:32:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427371+0000", 2024-04-26T04:32:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10588360600000001 2024-04-26T04:32:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427443+0000", 2024-04-26T04:32:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1688999999999998e-05 2024-04-26T04:32:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.859 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:478 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~688128] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:32:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.219546+0000", 2024-04-26T04:32:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 1.5534947780000001, 2024-04-26T04:32:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.60491518499999997, 2024-04-26T04:32:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.860 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 478 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219546+0000", 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219546+0000", 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219549+0000", 2024-04-26T04:32:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.322e-06 2024-04-26T04:32:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219983+0000", 2024-04-26T04:32:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00043452399999999999 2024-04-26T04:32:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.862 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219998+0000", 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5231999999999999e-05 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.220007+0000", 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9379999999999998e-06 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.320021+0000", 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100013729 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.863 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.864 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.864 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.320111+0000", 2024-04-26T04:32:51.864 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9907999999999994e-05 2024-04-26T04:32:51.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.864 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.864 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.864 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.320914+0000", 2024-04-26T04:32:51.864 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00080305000000000003 2024-04-26T04:32:51.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.864 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.864 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.864 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427530+0000", 2024-04-26T04:32:51.865 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.106616091 2024-04-26T04:32:51.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.865 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.865 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.865 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.824390+0000", 2024-04-26T04:32:51.865 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.396859621 2024-04-26T04:32:51.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.865 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.865 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.865 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.824461+0000", 2024-04-26T04:32:51.865 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1769999999999999e-05 2024-04-26T04:32:51.865 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.865 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:479 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~81920] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.221012+0000", 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 1.552029426, 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.70366377099999999, 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.866 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 479 2024-04-26T04:32:51.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.871 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.871 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.871 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221012+0000", 2024-04-26T04:32:51.871 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.871 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.871 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221012+0000", 2024-04-26T04:32:51.871 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.872 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.872 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.872 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221013+0000", 2024-04-26T04:32:51.872 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4750000000000001e-06 2024-04-26T04:32:51.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.872 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.872 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.872 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221057+0000", 2024-04-26T04:32:51.872 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.4305999999999999e-05 2024-04-26T04:32:51.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.872 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.872 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.872 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221069+0000", 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1783000000000001e-05 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221077+0000", 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7649999999999993e-06 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321785+0000", 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100707597 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.873 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.322104+0000", 2024-04-26T04:32:51.874 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00031923599999999998 2024-04-26T04:32:51.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.874 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.874 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.323441+0000", 2024-04-26T04:32:51.874 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001337233 2024-04-26T04:32:51.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.874 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.874 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427532+0000", 2024-04-26T04:32:51.874 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10409099199999999 2024-04-26T04:32:51.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924579+0000", 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49704692 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924675+0000", 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6464000000000003e-05 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:480 3.5s0 3:afb9923e:::smithi17829308-9:head [setxattr _header (56) in=7b] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:32:51.875 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:32:51.876 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 1.5519515369999999, 2024-04-26T04:32:51.876 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.70383227299999995, 2024-04-26T04:32:51.876 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.876 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.876 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:51.876 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.876 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.876 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.876 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 480 2024-04-26T04:32:51.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.876 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:32:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:32:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:32:51.877 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9999999999999997e-07 2024-04-26T04:32:51.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.878 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.878 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221093+0000", 2024-04-26T04:32:51.878 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.241e-06 2024-04-26T04:32:51.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.878 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.878 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221100+0000", 2024-04-26T04:32:51.878 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6080000000000001e-06 2024-04-26T04:32:51.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.879 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.879 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221106+0000", 2024-04-26T04:32:51.879 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4999999999999999e-06 2024-04-26T04:32:51.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.879 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.879 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.324423+0000", 2024-04-26T04:32:51.879 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103317706 2024-04-26T04:32:51.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.879 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.879 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.324652+0000", 2024-04-26T04:32:51.879 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022915899999999999 2024-04-26T04:32:51.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.880 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.880 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.325569+0000", 2024-04-26T04:32:51.880 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000916941 2024-04-26T04:32:51.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.880 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.880 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427565+0000", 2024-04-26T04:32:51.880 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101995135 2024-04-26T04:32:51.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.880 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924820+0000", 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49725566500000001 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924922+0000", 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000101618 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:481 3.5s0 3:afb9923e:::smithi17829308-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:32:51.881 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.221113+0000", 2024-04-26T04:32:51.882 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 1.551927764, 2024-04-26T04:32:51.882 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.80581624500000004, 2024-04-26T04:32:51.882 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.882 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.882 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:51.882 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:51.882 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:51.882 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:51.882 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 481 2024-04-26T04:32:51.882 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.882 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.882 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.883 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.883 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221113+0000", 2024-04-26T04:32:51.883 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.883 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.883 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221113+0000", 2024-04-26T04:32:51.883 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.883 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.883 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221114+0000", 2024-04-26T04:32:51.883 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.5700000000000002e-07 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221117+0000", 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.552e-06 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221125+0000", 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7710000000000006e-06 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.884 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321214+0000", 2024-04-26T04:32:51.885 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100088539 2024-04-26T04:32:51.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.885 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.885 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.326595+0000", 2024-04-26T04:32:51.885 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0053815479999999999 2024-04-26T04:32:51.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.885 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:51.885 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.326744+0000", 2024-04-26T04:32:51.885 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014871700000000001 2024-04-26T04:32:51.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.886 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.886 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.925438+0000", 2024-04-26T04:32:51.886 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.59869437999999997 2024-04-26T04:32:51.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.886 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:51.886 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.925572+0000", 2024-04-26T04:32:51.886 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000133912 2024-04-26T04:32:51.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.886 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.886 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.026930+0000", 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101357269 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.8s5 56/30 ECSubWrite(tid=72, reqid=client.4670.0:472, at_version=56'57, trim_to=0'0, roll_forward_to=52'56))", 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.286814+0000", 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 1.486226896, 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30298024000000001, 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:51.887 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.888 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.888 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.888 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.286814+0000", 2024-04-26T04:32:51.888 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.888 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.888 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.888 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.888 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.286814+0000", 2024-04-26T04:32:51.888 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.888 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.888 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.888 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.888 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.286815+0000", 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.13e-07 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.287036+0000", 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022085900000000001 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.287078+0000", 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.2579e-05 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.387175+0000", 2024-04-26T04:32:51.889 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100097065 2024-04-26T04:32:51.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.890 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.890 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.487423+0000", 2024-04-26T04:32:51.890 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100247791 2024-04-26T04:32:51.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.890 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.890 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.487440+0000", 2024-04-26T04:32:51.890 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6634e-05 2024-04-26T04:32:51.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.890 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.589711+0000", 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10227093299999999 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.589794+0000", 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.3866e-05 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.8s5 56/30 ECSubWrite(tid=73, reqid=client.4670.0:473, at_version=56'58, trim_to=0'0, roll_forward_to=52'56))", 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.387223+0000", 2024-04-26T04:32:51.891 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 1.385817952, 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20262782100000001, 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.387223+0000", 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.387223+0000", 2024-04-26T04:32:51.892 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.893 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.893 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.893 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.387224+0000", 2024-04-26T04:32:51.893 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3230000000000001e-06 2024-04-26T04:32:51.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.893 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.893 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.893 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.387415+0000", 2024-04-26T04:32:51.893 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000190566 2024-04-26T04:32:51.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.893 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.893 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.894 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.387433+0000", 2024-04-26T04:32:51.894 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7915999999999999e-05 2024-04-26T04:32:51.894 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.894 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.894 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.894 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.387444+0000", 2024-04-26T04:32:51.894 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.081e-05 2024-04-26T04:32:51.894 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.894 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.894 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.894 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.488217+0000", 2024-04-26T04:32:51.894 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10077359499999999 2024-04-26T04:32:51.894 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.488231+0000", 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3686000000000001e-05 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.589807+0000", 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101576149 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.589851+0000", 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3776000000000002e-05 2024-04-26T04:32:51.895 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.896 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.896 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.896 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.896 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.896 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.8s5 56/30 ECSubWrite(tid=74, reqid=client.4670.0:474, at_version=56'59, trim_to=0'0, roll_forward_to=52'56))", 2024-04-26T04:32:51.896 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.387461+0000", 2024-04-26T04:32:51.896 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 1.385579726, 2024-04-26T04:32:51.896 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20244387999999999, 2024-04-26T04:32:51.896 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:51.896 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:51.896 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:51.896 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:51.896 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.897 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:51.897 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.387461+0000", 2024-04-26T04:32:51.897 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.897 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.897 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:51.897 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.387461+0000", 2024-04-26T04:32:51.897 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:51.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.897 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.897 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:51.897 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.387462+0000", 2024-04-26T04:32:51.898 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.2e-07 2024-04-26T04:32:51.898 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.898 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.898 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:51.898 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.387535+0000", 2024-04-26T04:32:51.898 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.2945e-05 2024-04-26T04:32:51.898 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.898 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.898 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:51.898 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.387548+0000", 2024-04-26T04:32:51.898 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2835e-05 2024-04-26T04:32:51.898 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.898 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.387556+0000", 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4849999999999992e-06 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.488715+0000", 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101158758 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.488730+0000", 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5396999999999998e-05 2024-04-26T04:32:51.899 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.589860+0000", 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101130135 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.589905+0000", 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.4805000000000002e-05 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:51.900 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:51.902 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T04:32:51.903 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:51.900+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:52.003 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:52.001+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:52.036 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:52.036 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:32:52.036 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-04-26T04:32:52.036 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:52.051 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T04:32:52.104 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:52.101+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:52.169 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T04:32:52.170 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.14 to 1 (10000) 2024-04-26T04:32:52.177 INFO:tasks.rados.rados.0.smithi178.stdout:193: done (2 left) 2024-04-26T04:32:52.177 INFO:tasks.rados.rados.0.smithi178.stdout:194: rmattr oid 30 current snap is 12 2024-04-26T04:32:52.177 INFO:tasks.rados.rados.0.smithi178.stdout:194: done (2 left) 2024-04-26T04:32:52.177 INFO:tasks.rados.rados.0.smithi178.stdout:195: rmattr oid 27 current snap is 12 2024-04-26T04:32:52.177 INFO:tasks.rados.rados.0.smithi178.stdout:195: done (2 left) 2024-04-26T04:32:52.177 INFO:tasks.rados.rados.0.smithi178.stdout:196: rmattr oid 12 current snap is 12 2024-04-26T04:32:52.177 INFO:tasks.rados.rados.0.smithi178.stdout:196: done (2 left) 2024-04-26T04:32:52.177 INFO:tasks.rados.rados.0.smithi178.stdout:197: read oid 49 snap -1 2024-04-26T04:32:52.178 INFO:tasks.rados.rados.0.smithi178.stdout:197: expect (ObjNum 67 snap 3 seq_num 67) 2024-04-26T04:32:52.178 INFO:tasks.rados.rados.0.smithi178.stdout:198: read oid 50 snap -1 2024-04-26T04:32:52.178 INFO:tasks.rados.rados.0.smithi178.stdout:198: expect (ObjNum 71 snap 3 seq_num 71) 2024-04-26T04:32:52.178 INFO:tasks.rados.rados.0.smithi178.stdout:199: delete oid 19 current snap is 12 2024-04-26T04:32:52.178 INFO:tasks.rados.rados.0.smithi178.stdout:191: finishing write tid 1 to smithi17829308-2 2024-04-26T04:32:52.178 INFO:tasks.rados.rados.0.smithi178.stdout:192: finishing copy_from racing read to smithi17829308-38 2024-04-26T04:32:52.178 INFO:tasks.rados.rados.0.smithi178.stdout:191: finishing write tid 2 to smithi17829308-2 2024-04-26T04:32:52.178 INFO:tasks.rados.rados.0.smithi178.stdout:191: finishing write tid 3 to smithi17829308-2 2024-04-26T04:32:52.178 INFO:tasks.rados.rados.0.smithi178.stdout:191: finishing write tid 4 to smithi17829308-2 2024-04-26T04:32:52.178 INFO:tasks.rados.rados.0.smithi178.stdout:192: finishing copy_from to smithi17829308-38 2024-04-26T04:32:52.178 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 38 v 18 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-26T04:32:52.178 INFO:tasks.rados.rados.0.smithi178.stdout:191: finishing write tid 5 to smithi17829308-2 2024-04-26T04:32:52.178 INFO:tasks.rados.rados.0.smithi178.stdout:191: oid 2 updating version 0 to 60 2024-04-26T04:32:52.179 INFO:tasks.rados.rados.0.smithi178.stdout:191: oid 2 version 60 is already newer than 58 2024-04-26T04:32:52.179 INFO:tasks.rados.rados.0.smithi178.stdout:191: oid 2 version 60 is already newer than 59 2024-04-26T04:32:52.179 INFO:tasks.rados.rados.0.smithi178.stdout:191: oid 2 version 60 is already newer than 57 2024-04-26T04:32:52.179 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 2 v 60 (ObjNum 88 snap 12 seq_num 88) dirty exists 2024-04-26T04:32:52.179 INFO:tasks.rados.rados.0.smithi178.stdout:191: left oid 2 (ObjNum 88 snap 12 seq_num 88) 2024-04-26T04:32:52.204 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-26T04:32:52.205 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:52.202+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:52.206 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-04-26T04:32:52.227 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:52.227 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:32:52.227 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-04-26T04:32:52.227 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-04-26T04:32:52.228 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:52.246 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T04:32:52.305 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:52.300+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:52.341 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:52.341 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:52.366 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:52.367 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:52.383 INFO:tasks.rados.rados.0.smithi178.stdout:191: done (4 left) 2024-04-26T04:32:52.384 INFO:tasks.rados.rados.0.smithi178.stdout:192: done (3 left) 2024-04-26T04:32:52.384 INFO:tasks.rados.rados.0.smithi178.stdout:199: done (2 left) 2024-04-26T04:32:52.384 INFO:tasks.rados.rados.0.smithi178.stdout:200: read oid 9 snap -1 2024-04-26T04:32:52.384 INFO:tasks.rados.rados.0.smithi178.stdout:200: expect (ObjNum 87 snap 12 seq_num 87) 2024-04-26T04:32:52.384 INFO:tasks.rados.rados.0.smithi178.stdout:201: copy_from oid 38 from oid 7 current snap is 12 2024-04-26T04:32:52.384 INFO:tasks.rados.rados.0.smithi178.stdout:202: copy_from oid 31 from oid 32 current snap is 12 2024-04-26T04:32:52.384 INFO:tasks.rados.rados.0.smithi178.stdout:203: read oid 25 snap 9 2024-04-26T04:32:52.384 INFO:tasks.rados.rados.0.smithi178.stdout:203: expect deleted 2024-04-26T04:32:52.384 INFO:tasks.rados.rados.0.smithi178.stdout:204: delete oid 16 current snap is 12 2024-04-26T04:32:52.393 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:52.393 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:52.405 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:52.402+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:52.415 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:52.415 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:52.453 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:52.471 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T04:32:52.487 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:52.485+0000 7f205889d640 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap a after trimming was started 2024-04-26T04:32:52.492 INFO:tasks.rados.rados.0.smithi178.stdout:198: expect (ObjNum 71 snap 3 seq_num 71) 2024-04-26T04:32:52.508 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:52.504+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:52.600 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:32:52.601 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:32:52.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.601 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpRead(3.5s2 58/30 ECSubRead(tid=70, to_read={3:afb9923e:::smithi17829308-9:head=0,671744,0}, subchunks={3:afb9923e:::smithi17829308-9:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:52.601 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.489639+0000", 2024-04-26T04:32:52.601 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.109014362, 2024-04-26T04:32:52.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.109043892, 2024-04-26T04:32:52.601 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.601 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:52.601 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.489639+0000", 2024-04-26T04:32:52.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.489639+0000", 2024-04-26T04:32:52.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.489640+0000", 2024-04-26T04:32:52.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.3300000000000001e-07 2024-04-26T04:32:52.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.489648+0000", 2024-04-26T04:32:52.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5459999999999992e-06 2024-04-26T04:32:52.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.489664+0000", 2024-04-26T04:32:52.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5775999999999999e-05 2024-04-26T04:32:52.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.489676+0000", 2024-04-26T04:32:52.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1807999999999999e-05 2024-04-26T04:32:52.604 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.604 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.604 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.604 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpRead(3.5s2 58/30 ECSubRead(tid=71, to_read={3:afb9923e:::smithi17829308-9:head=0,671744,0}, subchunks={3:afb9923e:::smithi17829308-9:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:52.604 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.490685+0000", 2024-04-26T04:32:52.604 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.107967832, 2024-04-26T04:32:52.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.1080455, 2024-04-26T04:32:52.604 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.604 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:52.604 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.490685+0000", 2024-04-26T04:32:52.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.490685+0000", 2024-04-26T04:32:52.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.490686+0000", 2024-04-26T04:32:52.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7000000000000004e-07 2024-04-26T04:32:52.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.490694+0000", 2024-04-26T04:32:52.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1659999999999994e-06 2024-04-26T04:32:52.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.490711+0000", 2024-04-26T04:32:52.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6416e-05 2024-04-26T04:32:52.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.490721+0000", 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0783e-05 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpRead(3.5s2 58/30 ECSubRead(tid=72, to_read={3:afb9923e:::smithi17829308-9:head=0,671744,0}, subchunks={3:afb9923e:::smithi17829308-9:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.491458+0000", 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.107195399, 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10731455299999999, 2024-04-26T04:32:52.607 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.491458+0000", 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.491458+0000", 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.491458+0000", 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6499999999999998e-07 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.491468+0000", 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3859999999999996e-06 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.491485+0000", 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6762e-05 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.491496+0000", 2024-04-26T04:32:52.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.168e-05 2024-04-26T04:32:52.610 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.610 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.610 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.610 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpReadReply(3.bs0 58/30 ECSubReadReply(tid=196, attrs_read=0))", 2024-04-26T04:32:52.610 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.495900+0000", 2024-04-26T04:32:52.610 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.10275284699999999, 2024-04-26T04:32:52.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10291686999999999, 2024-04-26T04:32:52.610 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.610 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.610 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.495900+0000", 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.495900+0000", 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.495901+0000", 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1500000000000003e-07 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.496230+0000", 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000328468 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.496246+0000", 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6285000000000001e-05 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.496254+0000", 2024-04-26T04:32:52.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2970000000000004e-06 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:491 3.bs0 3:d909d6e5:::smithi17829308-49:head [read 0~1949696,checksum in=4b,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.176638+0000", 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.422014638, 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.42222292900000002, 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:52.613 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 491 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.176638+0000", 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.176638+0000", 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.176641+0000", 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4360000000000001e-06 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.176654+0000", 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3338e-05 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.176683+0000", 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8436e-05 2024-04-26T04:32:52.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.176696+0000", 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3689000000000001e-05 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.176786+0000", 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9161999999999998e-05 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op must wait for map", 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.176793+0000", 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3839999999999999e-06 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.377554+0000", 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20076127199999999 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.377681+0000", 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012667200000000001 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.617 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:492 3.bs0 3:d909d6e5:::smithi17829308-49:head [read 0~1949696,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.176730+0000", 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.42192324199999998, 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.42221756100000002, 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 492 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.176730+0000", 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.176730+0000", 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.176731+0000", 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6599999999999995e-07 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.176736+0000", 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4079999999999997e-06 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.176747+0000", 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1374e-05 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.276860+0000", 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10011286599999999 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.377047+0000", 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100186397 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.377055+0000", 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7940000000000008e-06 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478220+0000", 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101164133 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478404+0000", 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018452 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:493 3.bs0 3.a76b909b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.276896+0000", 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.32175661799999999, 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.32216058400000003, 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.622 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting_for_map not empty", 2024-04-26T04:32:52.623 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:52.623 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:52.623 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:52.623 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 493 2024-04-26T04:32:52.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.623 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.276896+0000", 2024-04-26T04:32:52.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.276896+0000", 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.276899+0000", 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4339999999999999e-06 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.276911+0000", 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1647999999999999e-05 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.276922+0000", 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1545999999999999e-05 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.276931+0000", 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1889999999999996e-06 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.377126+0000", 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100195086 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.377141+0000", 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4432e-05 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpReadReply(3.bs0 58/30 ECSubReadReply(tid=196, attrs_read=0))", 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.585214+0000", 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.013438722, 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.013933598, 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "no flag points reached", 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.585214+0000", 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.585214+0000", 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.585215+0000", 2024-04-26T04:32:52.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.108e-06 2024-04-26T04:32:52.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.585546+0000", 2024-04-26T04:32:52.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00033028300000000001 2024-04-26T04:32:52.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.585570+0000", 2024-04-26T04:32:52.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3802e-05 2024-04-26T04:32:52.628 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.628 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.1s0 58/30 ECSubWriteReply(tid=195, last_complete=58'62, committed=1, applied=1))", 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.598269+0000", 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.00038363500000000001, 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00090943000000000003, 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.598269+0000", 2024-04-26T04:32:52.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.598269+0000", 2024-04-26T04:32:52.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.598270+0000", 2024-04-26T04:32:52.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1200000000000002e-07 2024-04-26T04:32:52.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.598275+0000", 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.5850000000000001e-06 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.598287+0000", 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1950000000000001e-05 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.598294+0000", 2024-04-26T04:32:52.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5889999999999997e-06 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.598373+0000", 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9140000000000005e-05 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpReadReply(3.bs0 58/30 ECSubReadReply(tid=194, attrs_read=0))", 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.386141+0000", 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.21251207699999999, 2024-04-26T04:32:52.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.213076238, 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.386141+0000", 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.386141+0000", 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.386142+0000", 2024-04-26T04:32:52.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7899999999999998e-07 2024-04-26T04:32:52.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.387025+0000", 2024-04-26T04:32:52.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00088319400000000001 2024-04-26T04:32:52.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.387039+0000", 2024-04-26T04:32:52.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4129e-05 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.387046+0000", 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.2490000000000004e-06 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:503 3.7s0 3:e1f05b66:::smithi17829308-31:head [copy-from ver 45 in=55b] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.478827+0000", 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.119825893, 2024-04-26T04:32:52.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.120419655, 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 503 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478827+0000", 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478827+0000", 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478829+0000", 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0020000000000001e-06 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478838+0000", 2024-04-26T04:32:52.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9020000000000005e-06 2024-04-26T04:32:52.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478852+0000", 2024-04-26T04:32:52.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4389000000000001e-05 2024-04-26T04:32:52.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478862+0000", 2024-04-26T04:32:52.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8430000000000005e-06 2024-04-26T04:32:52.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:52.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478930+0000", 2024-04-26T04:32:52.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.7669000000000006e-05 2024-04-26T04:32:52.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:32:52.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.479049+0000", 2024-04-26T04:32:52.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011930399999999999 2024-04-26T04:32:52.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.639 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.639 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpRead(3.bs0 58/30 ECSubRead(tid=196, to_read={3:d909d6e5:::smithi17829308-49:head=0,487424,0}, subchunks={3:d909d6e5:::smithi17829308-49:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.478826+0000", 2024-04-26T04:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.119826839, 2024-04-26T04:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.120460474, 2024-04-26T04:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478826+0000", 2024-04-26T04:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478826+0000", 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478826+0000", 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478826+0000", 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478857+0000", 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0247999999999999e-05 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478869+0000", 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2187e-05 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.643 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:504 3.7s0 3:e1f05b66:::smithi17829308-31:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e58)", 2024-04-26T04:32:52.643 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.478884+0000", 2024-04-26T04:32:52.643 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.11976861599999999, 2024-04-26T04:32:52.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.120432635, 2024-04-26T04:32:52.643 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.643 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.643 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "waiting for blocked object", 2024-04-26T04:32:52.643 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:32:52.643 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:32:52.643 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:52.643 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 504 2024-04-26T04:32:52.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478884+0000", 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478884+0000", 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478885+0000", 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.9999999999999997e-07 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478888+0000", 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2679999999999999e-06 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478898+0000", 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6940000000000007e-06 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.478906+0000", 2024-04-26T04:32:52.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4420000000000001e-06 2024-04-26T04:32:52.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:52.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.479324+0000", 2024-04-26T04:32:52.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00041866199999999998 2024-04-26T04:32:52.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for blocked object", 2024-04-26T04:32:52.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.479387+0000", 2024-04-26T04:32:52.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2883000000000001e-05 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpReadReply(3.bs0 58/30 ECSubReadReply(tid=194, attrs_read=0))", 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.479194+0000", 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.119459387, 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.120159192, 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.479194+0000", 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.479194+0000", 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.479194+0000", 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.479194+0000", 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.479219+0000", 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5018e-05 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.479230+0000", 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1131999999999999e-05 2024-04-26T04:32:52.649 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.650 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.650 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.650 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpReadReply(3.bs0 58/30 ECSubReadReply(tid=196, attrs_read=0))", 2024-04-26T04:32:52.650 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.479804+0000", 2024-04-26T04:32:52.650 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.11884927000000001, 2024-04-26T04:32:52.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.119573578, 2024-04-26T04:32:52.650 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.650 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.650 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:32:52.650 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.479804+0000", 2024-04-26T04:32:52.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.479804+0000", 2024-04-26T04:32:52.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.479805+0000", 2024-04-26T04:32:52.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4399999999999999e-07 2024-04-26T04:32:52.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.480448+0000", 2024-04-26T04:32:52.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00064343899999999999 2024-04-26T04:32:52.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.480464+0000", 2024-04-26T04:32:52.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6035000000000001e-05 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.480472+0000", 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2309999999999996e-06 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.5s2 58/30 ECSubWrite(tid=68, reqid=osd.15.0:67, at_version=0'0, trim_to=0'0, roll_forward_to=58'48))", 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:52.489297+0000", 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.10935655900000001, 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.110119175, 2024-04-26T04:32:52.653 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:32:52.654 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:32:52.654 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:32:52.654 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:32:52.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:32:52.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.489297+0000", 2024-04-26T04:32:52.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:32:52.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.489297+0000", 2024-04-26T04:32:52.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:32:52.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:32:52.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.489297+0000", 2024-04-26T04:32:52.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.75e-07 2024-04-26T04:32:52.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:32:52.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.489311+0000", 2024-04-26T04:32:52.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3797e-05 2024-04-26T04:32:52.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:32:52.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.489321+0000", 2024-04-26T04:32:52.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6930000000000002e-06 2024-04-26T04:32:52.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:32:52.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.489330+0000", 2024-04-26T04:32:52.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.815e-06 2024-04-26T04:32:52.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:32:52.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.589533+0000", 2024-04-26T04:32:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10020237899999999 2024-04-26T04:32:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:32:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:32:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:32:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:52.589541+0000", 2024-04-26T04:32:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.1329999999999999e-06 2024-04-26T04:32:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:32:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:32:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:32:52.657 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 16 2024-04-26T04:32:52.657 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:32:52.659 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T04:32:52.660 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:52.657+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:52.690 INFO:tasks.rados.rados.0.smithi178.stdout:202: finishing copy_from racing read to smithi17829308-31 2024-04-26T04:32:52.701 INFO:tasks.rados.rados.0.smithi178.stdout:201: finishing copy_from racing read to smithi17829308-38 2024-04-26T04:32:52.760 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:52.761+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:52.788 INFO:tasks.rados.rados.0.smithi178.stdout:198: done (6 left) 2024-04-26T04:32:52.788 INFO:tasks.rados.rados.0.smithi178.stdout:203: done (5 left) 2024-04-26T04:32:52.788 INFO:tasks.rados.rados.0.smithi178.stdout:204: done (4 left) 2024-04-26T04:32:52.789 INFO:tasks.rados.rados.0.smithi178.stdout:205: rollback oid 8 current snap is 12 2024-04-26T04:32:52.789 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 8 to 9 2024-04-26T04:32:52.789 INFO:tasks.rados.rados.0.smithi178.stdout:206: delete oid 19 current snap is 12 2024-04-26T04:32:52.803 INFO:tasks.rados.rados.0.smithi178.stdout:202: finishing copy_from to smithi17829308-31 2024-04-26T04:32:52.803 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 31 v 45 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-04-26T04:32:52.807 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:52.807 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:52.830 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:52.830 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:52.831 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:52.831 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:52.845 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T04:32:52.856 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:52.856 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:52.861 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:52.861+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:52.880 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:52.880 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:52.889 INFO:tasks.rados.rados.0.smithi178.stdout:205: finishing rollback tid 1 to smithi17829308-8 2024-04-26T04:32:52.890 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 8 v 0 (ObjNum 177 snap 0 seq_num 2234958416) dirty dne 2024-04-26T04:32:52.909 INFO:tasks.rados.rados.0.smithi178.stdout:197: expect (ObjNum 67 snap 3 seq_num 67) 2024-04-26T04:32:52.962 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:52.961+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:52.963 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:52.963 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:52.978 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T04:32:53.019 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:53.019 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:53.025 INFO:tasks.rados.rados.0.smithi178.stdout:201: finishing copy_from to smithi17829308-38 2024-04-26T04:32:53.025 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 38 v 2 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-04-26T04:32:53.025 INFO:tasks.rados.rados.0.smithi178.stdout:200: expect (ObjNum 87 snap 12 seq_num 87) 2024-04-26T04:32:53.055 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:53.055 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:53.063 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:53.062+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:53.092 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:53.093 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:53.116 INFO:tasks.rados.rados.0.smithi178.stdout:197: done (5 left) 2024-04-26T04:32:53.116 INFO:tasks.rados.rados.0.smithi178.stdout:200: done (4 left) 2024-04-26T04:32:53.116 INFO:tasks.rados.rados.0.smithi178.stdout:201: done (3 left) 2024-04-26T04:32:53.116 INFO:tasks.rados.rados.0.smithi178.stdout:202: done (2 left) 2024-04-26T04:32:53.116 INFO:tasks.rados.rados.0.smithi178.stdout:205: done (1 left) 2024-04-26T04:32:53.116 INFO:tasks.rados.rados.0.smithi178.stdout:206: done (0 left) 2024-04-26T04:32:53.116 INFO:tasks.rados.rados.0.smithi178.stdout:207: copy_from oid 44 from oid 24 current snap is 12 2024-04-26T04:32:53.116 INFO:tasks.rados.rados.0.smithi178.stdout:208: rollback oid 13 current snap is 12 2024-04-26T04:32:53.116 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 13 to 9 2024-04-26T04:32:53.116 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 17 current snap is 12 2024-04-26T04:32:53.116 INFO:tasks.rados.rados.0.smithi178.stdout:209: seq_num 92 ranges {1212416=786432,1998848=491520,2490368=278528} 2024-04-26T04:32:53.124 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:53.124 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:53.149 INFO:tasks.rados.rados.0.smithi178.stdout:209: writing smithi17829308-17 from 1212416 to 1998848 tid 1 2024-04-26T04:32:53.154 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:53.155 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:53.155 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:53.157 INFO:tasks.rados.rados.0.smithi178.stdout:209: writing smithi17829308-17 from 1998848 to 2490368 tid 2 2024-04-26T04:32:53.162 INFO:tasks.rados.rados.0.smithi178.stdout:209: writing smithi17829308-17 from 2490368 to 2768896 tid 3 2024-04-26T04:32:53.162 INFO:tasks.rados.rados.0.smithi178.stdout:210: rollback oid 35 current snap is 12 2024-04-26T04:32:53.163 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 35 to 9 2024-04-26T04:32:53.163 INFO:tasks.rados.rados.0.smithi178.stdout:211: read oid 28 snap -1 2024-04-26T04:32:53.163 INFO:tasks.rados.rados.0.smithi178.stdout:211: expect (ObjNum 27 snap 0 seq_num 27) 2024-04-26T04:32:53.163 INFO:tasks.rados.rados.0.smithi178.stdout:212: rollback oid 50 current snap is 12 2024-04-26T04:32:53.163 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 50 to 9 2024-04-26T04:32:53.163 INFO:tasks.rados.rados.0.smithi178.stdout:213: delete oid 6 current snap is 12 2024-04-26T04:32:53.164 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:53.160+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:53.173 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T04:32:53.186 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:53.186 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:53.214 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:53.214 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:53.222 INFO:tasks.rados.rados.0.smithi178.stdout:207: finishing copy_from racing read to smithi17829308-44 2024-04-26T04:32:53.265 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:53.262+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:53.313 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:53.313 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:53.328 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T04:32:53.366 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:53.364+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:53.391 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:53.391 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:53.409 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-04-26T04:32:53.410 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.15 to 1 (10000) 2024-04-26T04:32:53.434 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-26T04:32:53.435 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:32:53.459 INFO:tasks.rados.rados.0.smithi178.stdout:209: finishing write tid 1 to smithi17829308-17 2024-04-26T04:32:53.467 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:53.466+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:53.472 INFO:tasks.rados.rados.0.smithi178.stdout:207: finishing copy_from to smithi17829308-44 2024-04-26T04:32:53.472 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 44 v 14 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-04-26T04:32:53.509 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:53.509 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:53.524 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T04:32:53.568 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:53.564+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:53.573 INFO:tasks.rados.rados.0.smithi178.stdout:211: expect (ObjNum 27 snap 0 seq_num 27) 2024-04-26T04:32:53.604 INFO:tasks.rados.rados.0.smithi178.stdout:212: finishing rollback tid 0 to smithi17829308-50 2024-04-26T04:32:53.605 INFO:tasks.rados.rados.0.smithi178.stdout:208: finishing rollback tid 0 to smithi17829308-13 2024-04-26T04:32:53.605 INFO:tasks.rados.rados.0.smithi178.stdout:208: finishing rollback tid 1 to smithi17829308-13 2024-04-26T04:32:53.606 INFO:tasks.rados.rados.0.smithi178.stdout:208: finishing rollback tid 2 to smithi17829308-13 2024-04-26T04:32:53.606 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 13 v 49 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-04-26T04:32:53.660 INFO:tasks.rados.rados.0.smithi178.stdout:209: finishing write tid 2 to smithi17829308-17 2024-04-26T04:32:53.661 INFO:tasks.rados.rados.0.smithi178.stdout:209: finishing write tid 3 to smithi17829308-17 2024-04-26T04:32:53.669 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:53.664+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:53.691 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:53.691 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:53.705 INFO:tasks.rados.rados.0.smithi178.stdout:212: finishing rollback tid 1 to smithi17829308-50 2024-04-26T04:32:53.705 INFO:tasks.rados.rados.0.smithi178.stdout:212: finishing rollback tid 2 to smithi17829308-50 2024-04-26T04:32:53.705 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 50 v 24 (ObjNum 71 snap 3 seq_num 71) dirty exists 2024-04-26T04:32:53.707 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T04:32:53.764 INFO:tasks.rados.rados.0.smithi178.stdout:209: finishing write tid 4 to smithi17829308-17 2024-04-26T04:32:53.770 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:53.769+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:53.799 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:53.800 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:32:53.815 INFO:tasks.rados.rados.0.smithi178.stdout:207: done (6 left) 2024-04-26T04:32:53.815 INFO:tasks.rados.rados.0.smithi178.stdout:208: done (5 left) 2024-04-26T04:32:53.815 INFO:tasks.rados.rados.0.smithi178.stdout:211: done (4 left) 2024-04-26T04:32:53.815 INFO:tasks.rados.rados.0.smithi178.stdout:212: done (3 left) 2024-04-26T04:32:53.815 INFO:tasks.rados.rados.0.smithi178.stdout:213: done (2 left) 2024-04-26T04:32:53.815 INFO:tasks.rados.rados.0.smithi178.stdout:214: read oid 43 snap -1 2024-04-26T04:32:53.816 INFO:tasks.rados.rados.0.smithi178.stdout:214: expect (ObjNum 34 snap 0 seq_num 34) 2024-04-26T04:32:53.816 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 50 current snap is 12 2024-04-26T04:32:53.816 INFO:tasks.rados.rados.0.smithi178.stdout:215: seq_num 94 ranges {2031616=425984,2457600=786432,3244032=376832} 2024-04-26T04:32:53.819 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-26T04:32:53.412600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":46531104,"num_objects":53,"num_object_clones":5,"num_object_copies":361,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":405,"num_read_kb":69552,"num_write":575,"num_write_kb":63412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":615,"ondisk_log_size":615,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":15,"num_per_pool_osds":15,"num_per_pool_omap_osds":15,"kb":1415577600,"kb_used":544416,"kb_used_data":89108,"kb_used_omap":23,"kb_used_meta":455016,"kb_avail":1415033184,"statfs":{"total":1449551462400,"available":1448993980416,"internally_reserved":0,"allocated":91246592,"data_stored":105726395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23926,"internal_metadata":465937034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6373376,"num_objects":-1,"num_object_clones":-1,"num_object_copies":-7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":-1,"num_read":29,"num_read_kb":24458,"num_write":24,"num_write_kb":8944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.076586"},"pg_stats":[{"pgid":"3.f","version":"52'38","reported_seq":172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T04:32:48.640211+0000","last_change":"2024-04-26T04:32:46.333051+0000","last_active":"2024-04-26T04:32:48.640211+0000","last_peered":"2024-04-26T04:32:48.640211+0000","last_clean":"2024-04-26T04:32:48.640211+0000","last_became_active":"2024-04-26T04:32:23.378967+0000","last_became_peered":"2024-04-26T04:32:23.378967+0000","last_unstale":"2024-04-26T04:32:48.640211+0000","last_undegraded":"2024-04-26T04:32:48.640211+0000","last_fullsized":"2024-04-26T04:32:48.640211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:47.266296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072528790000000003,"stat_sum":{"num_bytes":3522560,"num_objects":7,"num_object_clones":1,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":35,"num_read_kb":2953,"num_write":45,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,14,13,8,10,0],"acting":[11,1,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.e","version":"49'23","reported_seq":148,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.223178+0000","last_change":"2024-04-26T04:32:50.223919+0000","last_active":"2024-04-26T04:32:50.223178+0000","last_peered":"2024-04-26T04:32:50.223178+0000","last_clean":"2024-04-26T04:32:50.223178+0000","last_became_active":"2024-04-26T04:32:23.387938+0000","last_became_peered":"2024-04-26T04:32:23.387938+0000","last_unstale":"2024-04-26T04:32:50.223178+0000","last_undegraded":"2024-04-26T04:32:50.223178+0000","last_fullsized":"2024-04-26T04:32:50.223178+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:34.999991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001125551,"stat_sum":{"num_bytes":2736128,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":5,"num_write":14,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,9,1,11,8,6,12],"acting":[13,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"5","length":"5"},{"start":"d","length":"1"}]},{"pgid":"3.d","version":"52'34","reported_seq":164,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-26T04:32:49.279825+0000","last_change":"2024-04-26T04:32:46.540780+0000","last_active":"2024-04-26T04:32:49.279825+0000","last_peered":"2024-04-26T04:32:49.279825+0000","last_clean":"2024-04-26T04:32:49.279825+0000","last_became_active":"2024-04-26T04:32:23.392585+0000","last_became_peered":"2024-04-26T04:32:23.392585+0000","last_unstale":"2024-04-26T04:32:49.279825+0000","last_undegraded":"2024-04-26T04:32:49.279825+0000","last_fullsized":"2024-04-26T04:32:49.279825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:32.585139+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.373500371,"stat_sum":{"num_bytes":3457024,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":8025,"num_write":20,"num_write_kb":4336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.c","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T04:32:48.336825+0000","last_change":"2024-04-26T04:32:46.426919+0000","last_active":"2024-04-26T04:32:48.336825+0000","last_peered":"2024-04-26T04:32:48.336825+0000","last_clean":"2024-04-26T04:32:48.336825+0000","last_became_active":"2024-04-26T04:32:23.577814+0000","last_became_peered":"2024-04-26T04:32:23.577814+0000","last_unstale":"2024-04-26T04:32:48.336825+0000","last_undegraded":"2024-04-26T04:32:48.336825+0000","last_fullsized":"2024-04-26T04:32:48.336825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:32.148681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011162909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.b","version":"44'27","reported_seq":180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.683006+0000","last_change":"2024-04-26T04:32:52.683006+0000","last_active":"2024-04-26T04:32:52.683006+0000","last_peered":"2024-04-26T04:32:52.683006+0000","last_clean":"2024-04-26T04:32:52.683006+0000","last_became_active":"2024-04-26T04:32:23.220443+0000","last_became_peered":"2024-04-26T04:32:23.220443+0000","last_unstale":"2024-04-26T04:32:52.683006+0000","last_undegraded":"2024-04-26T04:32:52.683006+0000","last_fullsized":"2024-04-26T04:32:52.683006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:23.844223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.305636776,"stat_sum":{"num_bytes":3489792,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":31,"num_read_kb":15674,"num_write":17,"num_write_kb":5296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"3.a","version":"50'33","reported_seq":149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T04:32:48.279059+0000","last_change":"2024-04-26T04:32:46.176385+0000","last_active":"2024-04-26T04:32:48.279059+0000","last_peered":"2024-04-26T04:32:48.279059+0000","last_clean":"2024-04-26T04:32:48.279059+0000","last_became_active":"2024-04-26T04:32:23.310795+0000","last_became_peered":"2024-04-26T04:32:23.310795+0000","last_unstale":"2024-04-26T04:32:48.279059+0000","last_undegraded":"2024-04-26T04:32:48.279059+0000","last_fullsized":"2024-04-26T04:32:48.279059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:29.177502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014818399999999999,"stat_sum":{"num_bytes":294912,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":677,"num_write":15,"num_write_kb":2032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.9","version":"44'6","reported_seq":132,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.286077+0000","last_change":"2024-04-26T04:32:50.286936+0000","last_active":"2024-04-26T04:32:50.286077+0000","last_peered":"2024-04-26T04:32:50.286077+0000","last_clean":"2024-04-26T04:32:50.286077+0000","last_became_active":"2024-04-26T04:32:23.476584+0000","last_became_peered":"2024-04-26T04:32:23.476584+0000","last_unstale":"2024-04-26T04:32:50.286077+0000","last_undegraded":"2024-04-26T04:32:50.286077+0000","last_fullsized":"2024-04-26T04:32:50.286077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:32.261583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001337839,"stat_sum":{"num_bytes":458752,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":1378,"num_write":22,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"5","length":"5"},{"start":"d","length":"1"}]},{"pgid":"3.8","version":"56'60","reported_seq":188,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.287867+0000","last_change":"2024-04-26T04:32:50.396553+0000","last_active":"2024-04-26T04:32:50.287867+0000","last_peered":"2024-04-26T04:32:50.287867+0000","last_clean":"2024-04-26T04:32:50.287867+0000","last_became_active":"2024-04-26T04:32:23.487565+0000","last_became_peered":"2024-04-26T04:32:23.487565+0000","last_unstale":"2024-04-26T04:32:50.287867+0000","last_undegraded":"2024-04-26T04:32:50.287867+0000","last_fullsized":"2024-04-26T04:32:50.287867+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:51.116362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.11172868699999999,"stat_sum":{"num_bytes":5013504,"num_objects":7,"num_object_clones":0,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":28,"num_read_kb":8123,"num_write":67,"num_write_kb":5520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,12,1,14,15,6],"acting":[5,0,12,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5","length":"5"},{"start":"d","length":"1"}]},{"pgid":"1.0","version":"32'157","reported_seq":281,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.221407+0000","last_change":"2024-04-26T04:32:24.500132+0000","last_active":"2024-04-26T04:32:50.221407+0000","last_peered":"2024-04-26T04:32:50.221407+0000","last_clean":"2024-04-26T04:32:50.221407+0000","last_became_active":"2024-04-26T04:32:24.499360+0000","last_became_peered":"2024-04-26T04:32:24.499360+0000","last_unstale":"2024-04-26T04:32:50.221407+0000","last_undegraded":"2024-04-26T04:32:50.221407+0000","last_fullsized":"2024-04-26T04:32:50.221407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:32:08.365921+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.833702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3],"acting":[13,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.184607+0000","last_change":"2024-04-26T04:32:12.505010+0000","last_active":"2024-04-26T04:32:50.184607+0000","last_peered":"2024-04-26T04:32:50.184607+0000","last_clean":"2024-04-26T04:32:50.184607+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:32:50.184607+0000","last_undegraded":"2024-04-26T04:32:50.184607+0000","last_fullsized":"2024-04-26T04:32:50.184607+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.374592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"44'12","reported_seq":139,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.222571+0000","last_change":"2024-04-26T04:32:50.223263+0000","last_active":"2024-04-26T04:32:50.222571+0000","last_peered":"2024-04-26T04:32:50.222571+0000","last_clean":"2024-04-26T04:32:50.222571+0000","last_became_active":"2024-04-26T04:32:23.396007+0000","last_became_peered":"2024-04-26T04:32:23.396007+0000","last_unstale":"2024-04-26T04:32:50.222571+0000","last_undegraded":"2024-04-26T04:32:50.222571+0000","last_fullsized":"2024-04-26T04:32:50.222571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:43.249819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012434830000000001,"stat_sum":{"num_bytes":360448,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1060,"num_write":8,"num_write_kb":1584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10,3,6,15,1,11],"acting":[13,10,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"5","length":"5"},{"start":"d","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":66,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T04:32:48.278903+0000","last_change":"2024-04-26T04:32:12.603501+0000","last_active":"2024-04-26T04:32:48.278903+0000","last_peered":"2024-04-26T04:32:48.278903+0000","last_clean":"2024-04-26T04:32:48.278903+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:32:48.278903+0000","last_undegraded":"2024-04-26T04:32:48.278903+0000","last_fullsized":"2024-04-26T04:32:48.278903+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.945207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"58'62","reported_seq":204,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.598610+0000","last_change":"2024-04-26T04:32:52.599926+0000","last_active":"2024-04-26T04:32:52.598610+0000","last_peered":"2024-04-26T04:32:52.598610+0000","last_clean":"2024-04-26T04:32:52.598610+0000","last_became_active":"2024-04-26T04:32:23.399115+0000","last_became_peered":"2024-04-26T04:32:23.399115+0000","last_unstale":"2024-04-26T04:32:52.598610+0000","last_undegraded":"2024-04-26T04:32:52.598610+0000","last_fullsized":"2024-04-26T04:32:52.598610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:42.866003+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.32055450699999999,"stat_sum":{"num_bytes":9519104,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":829,"num_write":43,"num_write_kb":12080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,12,13],"acting":[4,11,9,5,10,12,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":69,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.221519+0000","last_change":"2024-04-26T04:32:12.505739+0000","last_active":"2024-04-26T04:32:50.221519+0000","last_peered":"2024-04-26T04:32:50.221519+0000","last_clean":"2024-04-26T04:32:50.221519+0000","last_became_active":"2024-04-26T04:32:12.504908+0000","last_became_peered":"2024-04-26T04:32:12.504908+0000","last_unstale":"2024-04-26T04:32:50.221519+0000","last_undegraded":"2024-04-26T04:32:50.221519+0000","last_fullsized":"2024-04-26T04:32:50.221519+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:41.080750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"44'6","reported_seq":130,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.286211+0000","last_change":"2024-04-26T04:32:50.286841+0000","last_active":"2024-04-26T04:32:50.286211+0000","last_peered":"2024-04-26T04:32:50.286211+0000","last_clean":"2024-04-26T04:32:50.286211+0000","last_became_active":"2024-04-26T04:32:23.386975+0000","last_became_peered":"2024-04-26T04:32:23.386975+0000","last_unstale":"2024-04-26T04:32:50.286211+0000","last_undegraded":"2024-04-26T04:32:50.286211+0000","last_fullsized":"2024-04-26T04:32:50.286211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:33.472829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011841130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":3,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,11,15,1,5,6,4],"acting":[14,11,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"5","length":"5"},{"start":"d","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":66,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T04:32:48.279383+0000","last_change":"2024-04-26T04:32:12.798464+0000","last_active":"2024-04-26T04:32:48.279383+0000","last_peered":"2024-04-26T04:32:48.279383+0000","last_clean":"2024-04-26T04:32:48.279383+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:32:48.279383+0000","last_undegraded":"2024-04-26T04:32:48.279383+0000","last_fullsized":"2024-04-26T04:32:48.279383+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.684262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":121,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.322451+0000","last_change":"2024-04-26T04:32:50.323174+0000","last_active":"2024-04-26T04:32:50.322451+0000","last_peered":"2024-04-26T04:32:50.322451+0000","last_clean":"2024-04-26T04:32:50.322451+0000","last_became_active":"2024-04-26T04:32:23.387136+0000","last_became_peered":"2024-04-26T04:32:23.387136+0000","last_unstale":"2024-04-26T04:32:50.322451+0000","last_undegraded":"2024-04-26T04:32:50.322451+0000","last_fullsized":"2024-04-26T04:32:50.322451+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:49.070433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001193639,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,6,5,3],"acting":[13,7,4,15,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"5","length":"5"},{"start":"d","length":"1"}]},{"pgid":"3.5","version":"58'48","reported_seq":197,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.391281+0000","last_change":"2024-04-26T04:32:52.391281+0000","last_active":"2024-04-26T04:32:52.391281+0000","last_peered":"2024-04-26T04:32:52.391281+0000","last_clean":"2024-04-26T04:32:52.391281+0000","last_became_active":"2024-04-26T04:32:23.295127+0000","last_became_peered":"2024-04-26T04:32:23.295127+0000","last_unstale":"2024-04-26T04:32:52.391281+0000","last_undegraded":"2024-04-26T04:32:52.391281+0000","last_fullsized":"2024-04-26T04:32:52.391281+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:31.359145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20557355699999999,"stat_sum":{"num_bytes":4964352,"num_objects":6,"num_object_clones":1,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":4715,"num_write":27,"num_write_kb":5712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.279450+0000","last_change":"2024-04-26T04:32:13.108823+0000","last_active":"2024-04-26T04:32:52.279450+0000","last_peered":"2024-04-26T04:32:52.279450+0000","last_clean":"2024-04-26T04:32:52.279450+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:32:52.279450+0000","last_undegraded":"2024-04-26T04:32:52.279450+0000","last_fullsized":"2024-04-26T04:32:52.279450+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"58'48","reported_seq":193,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.390782+0000","last_change":"2024-04-26T04:32:52.391430+0000","last_active":"2024-04-26T04:32:52.390782+0000","last_peered":"2024-04-26T04:32:52.390782+0000","last_clean":"2024-04-26T04:32:52.390782+0000","last_became_active":"2024-04-26T04:32:23.290645+0000","last_became_peered":"2024-04-26T04:32:23.290645+0000","last_unstale":"2024-04-26T04:32:52.390782+0000","last_undegraded":"2024-04-26T04:32:52.390782+0000","last_fullsized":"2024-04-26T04:32:52.390782+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:31.139484+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.205529189,"stat_sum":{"num_bytes":3653632,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":392,"num_write":20,"num_write_kb":4448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,4,1,0,14,13],"acting":[8,9,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":69,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.222346+0000","last_change":"2024-04-26T04:32:12.504400+0000","last_active":"2024-04-26T04:32:50.222346+0000","last_peered":"2024-04-26T04:32:50.222346+0000","last_clean":"2024-04-26T04:32:50.222346+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:32:50.222346+0000","last_undegraded":"2024-04-26T04:32:50.222346+0000","last_fullsized":"2024-04-26T04:32:50.222346+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.381954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"58'43","reported_seq":193,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.600658+0000","last_change":"2024-04-26T04:32:52.496520+0000","last_active":"2024-04-26T04:32:52.600658+0000","last_peered":"2024-04-26T04:32:52.600658+0000","last_clean":"2024-04-26T04:32:52.600658+0000","last_became_active":"2024-04-26T04:32:23.477983+0000","last_became_peered":"2024-04-26T04:32:23.477983+0000","last_unstale":"2024-04-26T04:32:52.600658+0000","last_undegraded":"2024-04-26T04:32:52.600658+0000","last_fullsized":"2024-04-26T04:32:52.600658+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:39.747597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.217132355,"stat_sum":{"num_bytes":5898240,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":15466,"num_write":22,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.285552+0000","last_change":"2024-04-26T04:32:12.697838+0000","last_active":"2024-04-26T04:32:52.285552+0000","last_peered":"2024-04-26T04:32:52.285552+0000","last_clean":"2024-04-26T04:32:52.285552+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:32:52.285552+0000","last_undegraded":"2024-04-26T04:32:52.285552+0000","last_fullsized":"2024-04-26T04:32:52.285552+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.961816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"52'18","reported_seq":166,"reported_epoch":58,"state":"active+clean+snaptrim","last_fresh":"2024-04-26T04:32:52.486896+0000","last_change":"2024-04-26T04:32:52.179660+0000","last_active":"2024-04-26T04:32:52.486896+0000","last_peered":"2024-04-26T04:32:52.486896+0000","last_clean":"2024-04-26T04:32:52.486896+0000","last_became_active":"2024-04-26T04:32:23.383105+0000","last_became_peered":"2024-04-26T04:32:23.383105+0000","last_unstale":"2024-04-26T04:32:52.486896+0000","last_undegraded":"2024-04-26T04:32:52.486896+0000","last_fullsized":"2024-04-26T04:32:52.486896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:22.359610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.100973428,"stat_sum":{"num_bytes":2572288,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":10136,"num_write":49,"num_write_kb":2512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"1"},{"start":"c","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.285420+0000","last_change":"2024-04-26T04:32:12.803651+0000","last_active":"2024-04-26T04:32:52.285420+0000","last_peered":"2024-04-26T04:32:52.285420+0000","last_clean":"2024-04-26T04:32:52.285420+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:32:52.285420+0000","last_undegraded":"2024-04-26T04:32:52.285420+0000","last_fullsized":"2024-04-26T04:32:52.285420+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:26.614652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":45940736,"num_objects":51,"num_object_clones":5,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":269,"num_read_kb":69434,"num_write":372,"num_write_kb":59952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":79941632,"data_stored":98897920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":458,"ondisk_log_size":458,"up":112,"acting":112,"num_store_stats":15},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1536000,"data_stored":1525344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247820,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38640,"kb_used_data":7456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333200,"statfs":{"total":96636764160,"available":96597196800,"internally_reserved":0,"allocated":7634944,"data_stored":9593417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247821,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":4368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":4472832,"data_stored":4317769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247820,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38388,"kb_used_data":7204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333452,"statfs":{"total":96636764160,"available":96597454848,"internally_reserved":0,"allocated":7376896,"data_stored":8281311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313228,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40368,"kb_used_data":9176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331472,"statfs":{"total":96636764160,"available":96595427328,"internally_reserved":0,"allocated":9396224,"data_stored":11103267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313228,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35580,"kb_used_data":8556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336260,"statfs":{"total":96636764160,"available":96600330240,"internally_reserved":0,"allocated":8761344,"data_stored":11414563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313228,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34164,"kb_used_data":7140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337676,"statfs":{"total":96636764160,"available":96601780224,"internally_reserved":0,"allocated":7311360,"data_stored":8588323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313228,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":5116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":5238784,"data_stored":6384675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313228,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31056,"kb_used_data":4032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340784,"statfs":{"total":96636764160,"available":96604962816,"internally_reserved":0,"allocated":4128768,"data_stored":4582435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":98784247821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280523,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29360,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342480,"statfs":{"total":96636764160,"available":96606699520,"internally_reserved":0,"allocated":2396160,"data_stored":2059323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280523,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41356,"kb_used_data":6016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330484,"statfs":{"total":96636764160,"available":96594415616,"internally_reserved":0,"allocated":6160384,"data_stored":7481883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247820,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37316,"kb_used_data":6132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334524,"statfs":{"total":96636764160,"available":96598552576,"internally_reserved":0,"allocated":6279168,"data_stored":6625983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5718016,"data_stored":7002815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247819,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37328,"kb_used_data":6140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334512,"statfs":{"total":96636764160,"available":96598540288,"internally_reserved":0,"allocated":6287360,"data_stored":7314111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280523,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39316,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332524,"statfs":{"total":96636764160,"available":96596504576,"internally_reserved":0,"allocated":4063232,"data_stored":4231753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247821,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":5880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":6021120,"data_stored":6744767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8609792,"data_stored":10899456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5525504,"data_stored":6217728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1142784,"data_stored":1142784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7745536,"data_stored":9351168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4964352,"data_stored":6594560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5840896,"data_stored":7213056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3325952,"data_stored":3833856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":6656000,"data_stored":8437760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":6897664,"data_stored":9195520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4452352,"data_stored":5955584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":5439488,"data_stored":7094272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":5459968,"data_stored":6529024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":6438912,"data_stored":7995392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":4100096,"data_stored":4284416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":3342336,"data_stored":4153344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:32:53.821 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:32:53.828 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:53.828 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:53.844 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T04:32:53.852 INFO:tasks.rados.rados.0.smithi178.stdout:215: writing smithi17829308-50 from 2031616 to 2457600 tid 1 2024-04-26T04:32:53.864 INFO:tasks.rados.rados.0.smithi178.stdout:215: writing smithi17829308-50 from 2457600 to 3244032 tid 2 2024-04-26T04:32:53.869 INFO:tasks.rados.rados.0.smithi178.stdout:215: writing smithi17829308-50 from 3244032 to 3620864 tid 3 2024-04-26T04:32:53.869 INFO:tasks.rados.rados.0.smithi178.stdout:216: rollback oid 38 current snap is 12 2024-04-26T04:32:53.869 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 38 to 9 2024-04-26T04:32:53.869 INFO:tasks.rados.rados.0.smithi178.stdout:209: finishing write tid 5 to smithi17829308-17 2024-04-26T04:32:53.869 INFO:tasks.rados.rados.0.smithi178.stdout:209: oid 17 updating version 0 to 64 2024-04-26T04:32:53.870 INFO:tasks.rados.rados.0.smithi178.stdout:209: oid 17 updating version 64 to 67 2024-04-26T04:32:53.870 INFO:tasks.rados.rados.0.smithi178.stdout:209: oid 17 version 67 is already newer than 65 2024-04-26T04:32:53.870 INFO:tasks.rados.rados.0.smithi178.stdout:209: oid 17 version 67 is already newer than 66 2024-04-26T04:32:53.870 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 17 v 67 (ObjNum 92 snap 12 seq_num 92) dirty exists 2024-04-26T04:32:53.870 INFO:tasks.rados.rados.0.smithi178.stdout:209: left oid 17 (ObjNum 92 snap 12 seq_num 92) 2024-04-26T04:32:53.870 INFO:tasks.rados.rados.0.smithi178.stdout:209: done (4 left) 2024-04-26T04:32:53.870 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 44 current snap is 12 2024-04-26T04:32:53.870 INFO:tasks.rados.rados.0.smithi178.stdout:217: seq_num 95 ranges {393216=442368,835584=540672,1376256=622592} 2024-04-26T04:32:53.871 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:53.868+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:53.879 INFO:tasks.rados.rados.0.smithi178.stdout:217: writing smithi17829308-44 from 393216 to 835584 tid 1 2024-04-26T04:32:53.885 INFO:tasks.rados.rados.0.smithi178.stdout:217: writing smithi17829308-44 from 835584 to 1376256 tid 2 2024-04-26T04:32:53.892 INFO:tasks.rados.rados.0.smithi178.stdout:217: writing smithi17829308-44 from 1376256 to 1998848 tid 3 2024-04-26T04:32:53.892 INFO:tasks.rados.rados.0.smithi178.stdout:218: delete oid 5 current snap is 12 2024-04-26T04:32:53.892 INFO:tasks.rados.rados.0.smithi178.stdout:210: finishing rollback tid 0 to smithi17829308-35 2024-04-26T04:32:53.971 INFO:tasks.rados.rados.0.smithi178.stdout:210: finishing rollback tid 1 to smithi17829308-35 2024-04-26T04:32:53.971 INFO:tasks.rados.rados.0.smithi178.stdout:210: finishing rollback tid 2 to smithi17829308-35 2024-04-26T04:32:53.971 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 35 v 64 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-04-26T04:32:53.972 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:53.969+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:54.023 INFO:tasks.rados.rados.0.smithi178.stdout:214: expect (ObjNum 34 snap 0 seq_num 34) 2024-04-26T04:32:54.038 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:54.038 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:54.053 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T04:32:54.073 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:54.068+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:54.170 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:32:54.171 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:32:54.174 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:54.173+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:54.177 INFO:tasks.rados.rados.0.smithi178.stdout:216: finishing rollback tid 0 to smithi17829308-38 2024-04-26T04:32:54.177 INFO:tasks.rados.rados.0.smithi178.stdout:216: finishing rollback tid 1 to smithi17829308-38 2024-04-26T04:32:54.187 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:54.188 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:54.188 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-26T04:32:53.412600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":46531104,"num_objects":53,"num_object_clones":5,"num_object_copies":361,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":405,"num_read_kb":69552,"num_write":575,"num_write_kb":63412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":615,"ondisk_log_size":615,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":15,"num_per_pool_osds":15,"num_per_pool_omap_osds":15,"kb":1415577600,"kb_used":544416,"kb_used_data":89108,"kb_used_omap":23,"kb_used_meta":455016,"kb_avail":1415033184,"statfs":{"total":1449551462400,"available":1448993980416,"internally_reserved":0,"allocated":91246592,"data_stored":105726395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23926,"internal_metadata":465937034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6373376,"num_objects":-1,"num_object_clones":-1,"num_object_copies":-7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":-1,"num_read":29,"num_read_kb":24458,"num_write":24,"num_write_kb":8944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.076586"},"pg_stats":[{"pgid":"3.f","version":"52'38","reported_seq":172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T04:32:48.640211+0000","last_change":"2024-04-26T04:32:46.333051+0000","last_active":"2024-04-26T04:32:48.640211+0000","last_peered":"2024-04-26T04:32:48.640211+0000","last_clean":"2024-04-26T04:32:48.640211+0000","last_became_active":"2024-04-26T04:32:23.378967+0000","last_became_peered":"2024-04-26T04:32:23.378967+0000","last_unstale":"2024-04-26T04:32:48.640211+0000","last_undegraded":"2024-04-26T04:32:48.640211+0000","last_fullsized":"2024-04-26T04:32:48.640211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:47.266296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072528790000000003,"stat_sum":{"num_bytes":3522560,"num_objects":7,"num_object_clones":1,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":35,"num_read_kb":2953,"num_write":45,"num_write_kb":6016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,14,13,8,10,0],"acting":[11,1,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.e","version":"49'23","reported_seq":148,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.223178+0000","last_change":"2024-04-26T04:32:50.223919+0000","last_active":"2024-04-26T04:32:50.223178+0000","last_peered":"2024-04-26T04:32:50.223178+0000","last_clean":"2024-04-26T04:32:50.223178+0000","last_became_active":"2024-04-26T04:32:23.387938+0000","last_became_peered":"2024-04-26T04:32:23.387938+0000","last_unstale":"2024-04-26T04:32:50.223178+0000","last_undegraded":"2024-04-26T04:32:50.223178+0000","last_fullsized":"2024-04-26T04:32:50.223178+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:34.999991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001125551,"stat_sum":{"num_bytes":2736128,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":9,"num_read_kb":5,"num_write":14,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,9,1,11,8,6,12],"acting":[13,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"5","length":"5"},{"start":"d","length":"1"}]},{"pgid":"3.d","version":"52'34","reported_seq":164,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-26T04:32:49.279825+0000","last_change":"2024-04-26T04:32:46.540780+0000","last_active":"2024-04-26T04:32:49.279825+0000","last_peered":"2024-04-26T04:32:49.279825+0000","last_clean":"2024-04-26T04:32:49.279825+0000","last_became_active":"2024-04-26T04:32:23.392585+0000","last_became_peered":"2024-04-26T04:32:23.392585+0000","last_unstale":"2024-04-26T04:32:49.279825+0000","last_undegraded":"2024-04-26T04:32:49.279825+0000","last_fullsized":"2024-04-26T04:32:49.279825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:32.585139+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.373500371,"stat_sum":{"num_bytes":3457024,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":8025,"num_write":20,"num_write_kb":4336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.c","version":"0'0","reported_seq":113,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T04:32:48.336825+0000","last_change":"2024-04-26T04:32:46.426919+0000","last_active":"2024-04-26T04:32:48.336825+0000","last_peered":"2024-04-26T04:32:48.336825+0000","last_clean":"2024-04-26T04:32:48.336825+0000","last_became_active":"2024-04-26T04:32:23.577814+0000","last_became_peered":"2024-04-26T04:32:23.577814+0000","last_unstale":"2024-04-26T04:32:48.336825+0000","last_undegraded":"2024-04-26T04:32:48.336825+0000","last_fullsized":"2024-04-26T04:32:48.336825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:32.148681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011162909999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.b","version":"44'27","reported_seq":180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.683006+0000","last_change":"2024-04-26T04:32:52.683006+0000","last_active":"2024-04-26T04:32:52.683006+0000","last_peered":"2024-04-26T04:32:52.683006+0000","last_clean":"2024-04-26T04:32:52.683006+0000","last_became_active":"2024-04-26T04:32:23.220443+0000","last_became_peered":"2024-04-26T04:32:23.220443+0000","last_unstale":"2024-04-26T04:32:52.683006+0000","last_undegraded":"2024-04-26T04:32:52.683006+0000","last_fullsized":"2024-04-26T04:32:52.683006+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:23.844223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.305636776,"stat_sum":{"num_bytes":3489792,"num_objects":4,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":31,"num_read_kb":15674,"num_write":17,"num_write_kb":5296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"3.a","version":"50'33","reported_seq":149,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T04:32:48.279059+0000","last_change":"2024-04-26T04:32:46.176385+0000","last_active":"2024-04-26T04:32:48.279059+0000","last_peered":"2024-04-26T04:32:48.279059+0000","last_clean":"2024-04-26T04:32:48.279059+0000","last_became_active":"2024-04-26T04:32:23.310795+0000","last_became_peered":"2024-04-26T04:32:23.310795+0000","last_unstale":"2024-04-26T04:32:48.279059+0000","last_undegraded":"2024-04-26T04:32:48.279059+0000","last_fullsized":"2024-04-26T04:32:48.279059+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:29.177502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014818399999999999,"stat_sum":{"num_bytes":294912,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":677,"num_write":15,"num_write_kb":2032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"5","length":"5"}]},{"pgid":"3.9","version":"44'6","reported_seq":132,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.286077+0000","last_change":"2024-04-26T04:32:50.286936+0000","last_active":"2024-04-26T04:32:50.286077+0000","last_peered":"2024-04-26T04:32:50.286077+0000","last_clean":"2024-04-26T04:32:50.286077+0000","last_became_active":"2024-04-26T04:32:23.476584+0000","last_became_peered":"2024-04-26T04:32:23.476584+0000","last_unstale":"2024-04-26T04:32:50.286077+0000","last_undegraded":"2024-04-26T04:32:50.286077+0000","last_fullsized":"2024-04-26T04:32:50.286077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:32.261583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001337839,"stat_sum":{"num_bytes":458752,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":1378,"num_write":22,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"5","length":"5"},{"start":"d","length":"1"}]},{"pgid":"3.8","version":"56'60","reported_seq":188,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.287867+0000","last_change":"2024-04-26T04:32:50.396553+0000","last_active":"2024-04-26T04:32:50.287867+0000","last_peered":"2024-04-26T04:32:50.287867+0000","last_clean":"2024-04-26T04:32:50.287867+0000","last_became_active":"2024-04-26T04:32:23.487565+0000","last_became_peered":"2024-04-26T04:32:23.487565+0000","last_unstale":"2024-04-26T04:32:50.287867+0000","last_undegraded":"2024-04-26T04:32:50.287867+0000","last_fullsized":"2024-04-26T04:32:50.287867+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:51.116362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.11172868699999999,"stat_sum":{"num_bytes":5013504,"num_objects":7,"num_object_clones":0,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":28,"num_read_kb":8123,"num_write":67,"num_write_kb":5520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,12,1,14,15,6],"acting":[5,0,12,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"5","length":"5"},{"start":"d","length":"1"}]},{"pgid":"1.0","version":"32'157","reported_seq":281,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.221407+0000","last_change":"2024-04-26T04:32:24.500132+0000","last_active":"2024-04-26T04:32:50.221407+0000","last_peered":"2024-04-26T04:32:50.221407+0000","last_clean":"2024-04-26T04:32:50.221407+0000","last_became_active":"2024-04-26T04:32:24.499360+0000","last_became_peered":"2024-04-26T04:32:24.499360+0000","last_unstale":"2024-04-26T04:32:50.221407+0000","last_undegraded":"2024-04-26T04:32:50.221407+0000","last_fullsized":"2024-04-26T04:32:50.221407+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:32:08.365921+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.833702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3],"acting":[13,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.184607+0000","last_change":"2024-04-26T04:32:12.505010+0000","last_active":"2024-04-26T04:32:50.184607+0000","last_peered":"2024-04-26T04:32:50.184607+0000","last_clean":"2024-04-26T04:32:50.184607+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:32:50.184607+0000","last_undegraded":"2024-04-26T04:32:50.184607+0000","last_fullsized":"2024-04-26T04:32:50.184607+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.374592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"44'12","reported_seq":139,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.222571+0000","last_change":"2024-04-26T04:32:50.223263+0000","last_active":"2024-04-26T04:32:50.222571+0000","last_peered":"2024-04-26T04:32:50.222571+0000","last_clean":"2024-04-26T04:32:50.222571+0000","last_became_active":"2024-04-26T04:32:23.396007+0000","last_became_peered":"2024-04-26T04:32:23.396007+0000","last_unstale":"2024-04-26T04:32:50.222571+0000","last_undegraded":"2024-04-26T04:32:50.222571+0000","last_fullsized":"2024-04-26T04:32:50.222571+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:43.249819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012434830000000001,"stat_sum":{"num_bytes":360448,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":1060,"num_write":8,"num_write_kb":1584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10,3,6,15,1,11],"acting":[13,10,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"5","length":"5"},{"start":"d","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":66,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T04:32:48.278903+0000","last_change":"2024-04-26T04:32:12.603501+0000","last_active":"2024-04-26T04:32:48.278903+0000","last_peered":"2024-04-26T04:32:48.278903+0000","last_clean":"2024-04-26T04:32:48.278903+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:32:48.278903+0000","last_undegraded":"2024-04-26T04:32:48.278903+0000","last_fullsized":"2024-04-26T04:32:48.278903+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.945207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"58'62","reported_seq":204,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.598610+0000","last_change":"2024-04-26T04:32:52.599926+0000","last_active":"2024-04-26T04:32:52.598610+0000","last_peered":"2024-04-26T04:32:52.598610+0000","last_clean":"2024-04-26T04:32:52.598610+0000","last_became_active":"2024-04-26T04:32:23.399115+0000","last_became_peered":"2024-04-26T04:32:23.399115+0000","last_unstale":"2024-04-26T04:32:52.598610+0000","last_undegraded":"2024-04-26T04:32:52.598610+0000","last_fullsized":"2024-04-26T04:32:52.598610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:42.866003+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.32055450699999999,"stat_sum":{"num_bytes":9519104,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":23,"num_read_kb":829,"num_write":43,"num_write_kb":12080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,12,13],"acting":[4,11,9,5,10,12,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":69,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.221519+0000","last_change":"2024-04-26T04:32:12.505739+0000","last_active":"2024-04-26T04:32:50.221519+0000","last_peered":"2024-04-26T04:32:50.221519+0000","last_clean":"2024-04-26T04:32:50.221519+0000","last_became_active":"2024-04-26T04:32:12.504908+0000","last_became_peered":"2024-04-26T04:32:12.504908+0000","last_unstale":"2024-04-26T04:32:50.221519+0000","last_undegraded":"2024-04-26T04:32:50.221519+0000","last_fullsized":"2024-04-26T04:32:50.221519+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:41.080750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"44'6","reported_seq":130,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.286211+0000","last_change":"2024-04-26T04:32:50.286841+0000","last_active":"2024-04-26T04:32:50.286211+0000","last_peered":"2024-04-26T04:32:50.286211+0000","last_clean":"2024-04-26T04:32:50.286211+0000","last_became_active":"2024-04-26T04:32:23.386975+0000","last_became_peered":"2024-04-26T04:32:23.386975+0000","last_unstale":"2024-04-26T04:32:50.286211+0000","last_undegraded":"2024-04-26T04:32:50.286211+0000","last_fullsized":"2024-04-26T04:32:50.286211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:33.472829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011841130000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":3,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,11,15,1,5,6,4],"acting":[14,11,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"5","length":"5"},{"start":"d","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":66,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T04:32:48.279383+0000","last_change":"2024-04-26T04:32:12.798464+0000","last_active":"2024-04-26T04:32:48.279383+0000","last_peered":"2024-04-26T04:32:48.279383+0000","last_clean":"2024-04-26T04:32:48.279383+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:32:48.279383+0000","last_undegraded":"2024-04-26T04:32:48.279383+0000","last_fullsized":"2024-04-26T04:32:48.279383+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.684262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":121,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.322451+0000","last_change":"2024-04-26T04:32:50.323174+0000","last_active":"2024-04-26T04:32:50.322451+0000","last_peered":"2024-04-26T04:32:50.322451+0000","last_clean":"2024-04-26T04:32:50.322451+0000","last_became_active":"2024-04-26T04:32:23.387136+0000","last_became_peered":"2024-04-26T04:32:23.387136+0000","last_unstale":"2024-04-26T04:32:50.322451+0000","last_undegraded":"2024-04-26T04:32:50.322451+0000","last_fullsized":"2024-04-26T04:32:50.322451+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:49.070433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001193639,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,6,5,3],"acting":[13,7,4,15,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"5","length":"5"},{"start":"d","length":"1"}]},{"pgid":"3.5","version":"58'48","reported_seq":197,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.391281+0000","last_change":"2024-04-26T04:32:52.391281+0000","last_active":"2024-04-26T04:32:52.391281+0000","last_peered":"2024-04-26T04:32:52.391281+0000","last_clean":"2024-04-26T04:32:52.391281+0000","last_became_active":"2024-04-26T04:32:23.295127+0000","last_became_peered":"2024-04-26T04:32:23.295127+0000","last_unstale":"2024-04-26T04:32:52.391281+0000","last_undegraded":"2024-04-26T04:32:52.391281+0000","last_fullsized":"2024-04-26T04:32:52.391281+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:31.359145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20557355699999999,"stat_sum":{"num_bytes":4964352,"num_objects":6,"num_object_clones":1,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":4715,"num_write":27,"num_write_kb":5712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":72,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.279450+0000","last_change":"2024-04-26T04:32:13.108823+0000","last_active":"2024-04-26T04:32:52.279450+0000","last_peered":"2024-04-26T04:32:52.279450+0000","last_clean":"2024-04-26T04:32:52.279450+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:32:52.279450+0000","last_undegraded":"2024-04-26T04:32:52.279450+0000","last_fullsized":"2024-04-26T04:32:52.279450+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"58'48","reported_seq":193,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.390782+0000","last_change":"2024-04-26T04:32:52.391430+0000","last_active":"2024-04-26T04:32:52.390782+0000","last_peered":"2024-04-26T04:32:52.390782+0000","last_clean":"2024-04-26T04:32:52.390782+0000","last_became_active":"2024-04-26T04:32:23.290645+0000","last_became_peered":"2024-04-26T04:32:23.290645+0000","last_unstale":"2024-04-26T04:32:52.390782+0000","last_undegraded":"2024-04-26T04:32:52.390782+0000","last_fullsized":"2024-04-26T04:32:52.390782+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:31.139484+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.205529189,"stat_sum":{"num_bytes":3653632,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":392,"num_write":20,"num_write_kb":4448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,4,1,0,14,13],"acting":[8,9,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":69,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T04:32:50.222346+0000","last_change":"2024-04-26T04:32:12.504400+0000","last_active":"2024-04-26T04:32:50.222346+0000","last_peered":"2024-04-26T04:32:50.222346+0000","last_clean":"2024-04-26T04:32:50.222346+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:32:50.222346+0000","last_undegraded":"2024-04-26T04:32:50.222346+0000","last_fullsized":"2024-04-26T04:32:50.222346+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.381954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"58'43","reported_seq":193,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.600658+0000","last_change":"2024-04-26T04:32:52.496520+0000","last_active":"2024-04-26T04:32:52.600658+0000","last_peered":"2024-04-26T04:32:52.600658+0000","last_clean":"2024-04-26T04:32:52.600658+0000","last_became_active":"2024-04-26T04:32:23.477983+0000","last_became_peered":"2024-04-26T04:32:23.477983+0000","last_unstale":"2024-04-26T04:32:52.600658+0000","last_undegraded":"2024-04-26T04:32:52.600658+0000","last_fullsized":"2024-04-26T04:32:52.600658+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:39.747597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.217132355,"stat_sum":{"num_bytes":5898240,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":15466,"num_write":22,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.285552+0000","last_change":"2024-04-26T04:32:12.697838+0000","last_active":"2024-04-26T04:32:52.285552+0000","last_peered":"2024-04-26T04:32:52.285552+0000","last_clean":"2024-04-26T04:32:52.285552+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:32:52.285552+0000","last_undegraded":"2024-04-26T04:32:52.285552+0000","last_fullsized":"2024-04-26T04:32:52.285552+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.961816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"52'18","reported_seq":166,"reported_epoch":58,"state":"active+clean+snaptrim","last_fresh":"2024-04-26T04:32:52.486896+0000","last_change":"2024-04-26T04:32:52.179660+0000","last_active":"2024-04-26T04:32:52.486896+0000","last_peered":"2024-04-26T04:32:52.486896+0000","last_clean":"2024-04-26T04:32:52.486896+0000","last_became_active":"2024-04-26T04:32:23.383105+0000","last_became_peered":"2024-04-26T04:32:23.383105+0000","last_unstale":"2024-04-26T04:32:52.486896+0000","last_undegraded":"2024-04-26T04:32:52.486896+0000","last_fullsized":"2024-04-26T04:32:52.486896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:22.359610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.100973428,"stat_sum":{"num_bytes":2572288,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":10136,"num_write":49,"num_write_kb":2512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"1"},{"start":"c","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":73,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T04:32:52.285420+0000","last_change":"2024-04-26T04:32:12.803651+0000","last_active":"2024-04-26T04:32:52.285420+0000","last_peered":"2024-04-26T04:32:52.285420+0000","last_clean":"2024-04-26T04:32:52.285420+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:32:52.285420+0000","last_undegraded":"2024-04-26T04:32:52.285420+0000","last_fullsized":"2024-04-26T04:32:52.285420+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:26.614652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":45940736,"num_objects":51,"num_object_clones":5,"num_object_copies":357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":269,"num_read_kb":69434,"num_write":372,"num_write_kb":59952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":79941632,"data_stored":98897920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":458,"ondisk_log_size":458,"up":112,"acting":112,"num_store_stats":15},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1536000,"data_stored":1525344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247820,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38640,"kb_used_data":7456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333200,"statfs":{"total":96636764160,"available":96597196800,"internally_reserved":0,"allocated":7634944,"data_stored":9593417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247821,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":4368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":4472832,"data_stored":4317769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247820,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38388,"kb_used_data":7204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333452,"statfs":{"total":96636764160,"available":96597454848,"internally_reserved":0,"allocated":7376896,"data_stored":8281311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313228,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40368,"kb_used_data":9176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331472,"statfs":{"total":96636764160,"available":96595427328,"internally_reserved":0,"allocated":9396224,"data_stored":11103267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313228,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35580,"kb_used_data":8556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336260,"statfs":{"total":96636764160,"available":96600330240,"internally_reserved":0,"allocated":8761344,"data_stored":11414563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313228,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34164,"kb_used_data":7140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337676,"statfs":{"total":96636764160,"available":96601780224,"internally_reserved":0,"allocated":7311360,"data_stored":8588323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313228,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":5116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":5238784,"data_stored":6384675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313228,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31056,"kb_used_data":4032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340784,"statfs":{"total":96636764160,"available":96604962816,"internally_reserved":0,"allocated":4128768,"data_stored":4582435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":98784247821,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280523,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29360,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342480,"statfs":{"total":96636764160,"available":96606699520,"internally_reserved":0,"allocated":2396160,"data_stored":2059323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280523,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41356,"kb_used_data":6016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330484,"statfs":{"total":96636764160,"available":96594415616,"internally_reserved":0,"allocated":6160384,"data_stored":7481883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247820,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37316,"kb_used_data":6132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334524,"statfs":{"total":96636764160,"available":96598552576,"internally_reserved":0,"allocated":6279168,"data_stored":6625983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247821,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36776,"kb_used_data":5584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335064,"statfs":{"total":96636764160,"available":96599105536,"internally_reserved":0,"allocated":5718016,"data_stored":7002815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247819,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37328,"kb_used_data":6140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334512,"statfs":{"total":96636764160,"available":96598540288,"internally_reserved":0,"allocated":6287360,"data_stored":7314111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280523,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39316,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332524,"statfs":{"total":96636764160,"available":96596504576,"internally_reserved":0,"allocated":4063232,"data_stored":4231753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247821,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37068,"kb_used_data":5880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334772,"statfs":{"total":96636764160,"available":96598806528,"internally_reserved":0,"allocated":6021120,"data_stored":6744767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8609792,"data_stored":10899456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5525504,"data_stored":6217728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1142784,"data_stored":1142784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7745536,"data_stored":9351168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4964352,"data_stored":6594560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5840896,"data_stored":7213056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3325952,"data_stored":3833856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":6656000,"data_stored":8437760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":6897664,"data_stored":9195520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4452352,"data_stored":5955584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":5439488,"data_stored":7094272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":5459968,"data_stored":6529024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":6438912,"data_stored":7995392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":4100096,"data_stored":4284416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":3342336,"data_stored":4153344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:32:54.191 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-26T04:32:54.191 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T04:32:54.192 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-04-26T04:32:54.192 INFO:tasks.thrashosds.thrasher:Setting osd 13 primary_affinity to 0.443890 2024-04-26T04:32:54.192 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 13 0.4438898176264633 2024-04-26T04:32:54.199 INFO:tasks.rados.rados.0.smithi178.stdout:217: finishing write tid 1 to smithi17829308-44 2024-04-26T04:32:54.201 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T04:32:54.275 INFO:tasks.rados.rados.0.smithi178.stdout:215: finishing write tid 1 to smithi17829308-50 2024-04-26T04:32:54.275 INFO:tasks.rados.rados.0.smithi178.stdout:215: finishing write tid 2 to smithi17829308-50 2024-04-26T04:32:54.275 INFO:tasks.rados.rados.0.smithi178.stdout:215: finishing write tid 3 to smithi17829308-50 2024-04-26T04:32:54.275 INFO:tasks.rados.rados.0.smithi178.stdout:215: finishing write tid 4 to smithi17829308-50 2024-04-26T04:32:54.275 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:54.272+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:54.277 INFO:tasks.rados.rados.0.smithi178.stdout:217: finishing write tid 2 to smithi17829308-44 2024-04-26T04:32:54.278 INFO:tasks.rados.rados.0.smithi178.stdout:217: finishing write tid 3 to smithi17829308-44 2024-04-26T04:32:54.278 INFO:tasks.rados.rados.0.smithi178.stdout:217: finishing write tid 4 to smithi17829308-44 2024-04-26T04:32:54.368 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:54.369 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:54.375 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:54.374+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:54.376 INFO:tasks.rados.rados.0.smithi178.stdout:210: done (5 left) 2024-04-26T04:32:54.377 INFO:tasks.rados.rados.0.smithi178.stdout:214: done (4 left) 2024-04-26T04:32:54.377 INFO:tasks.rados.rados.0.smithi178.stdout:218: done (3 left) 2024-04-26T04:32:54.377 INFO:tasks.rados.rados.0.smithi178.stdout:219: read oid 29 snap -1 2024-04-26T04:32:54.377 INFO:tasks.rados.rados.0.smithi178.stdout:219: expect (ObjNum 55 snap 1 seq_num 55) 2024-04-26T04:32:54.377 INFO:tasks.rados.rados.0.smithi178.stdout:220: read oid 35 snap 9 2024-04-26T04:32:54.377 INFO:tasks.rados.rados.0.smithi178.stdout:220: expect (ObjNum 34 snap 0 seq_num 34) 2024-04-26T04:32:54.377 INFO:tasks.rados.rados.0.smithi178.stdout:221: read oid 34 snap -1 2024-04-26T04:32:54.377 INFO:tasks.rados.rados.0.smithi178.stdout:221: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-26T04:32:54.377 INFO:tasks.rados.rados.0.smithi178.stdout:222: delete oid 27 current snap is 12 2024-04-26T04:32:54.384 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T04:32:54.476 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:54.472+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:54.479 INFO:tasks.rados.rados.0.smithi178.stdout:216: finishing rollback tid 2 to smithi17829308-38 2024-04-26T04:32:54.480 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 38 v 54 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-04-26T04:32:54.481 INFO:tasks.rados.rados.0.smithi178.stdout:216: done (6 left) 2024-04-26T04:32:54.482 INFO:tasks.rados.rados.0.smithi178.stdout:222: done (5 left) 2024-04-26T04:32:54.482 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 8 current snap is 12 2024-04-26T04:32:54.482 INFO:tasks.rados.rados.0.smithi178.stdout:223: seq_num 98 ranges {0=491520,491520=589824,1081344=557056,1638400=16384} 2024-04-26T04:32:54.490 INFO:tasks.rados.rados.0.smithi178.stdout:223: writing smithi17829308-8 from 0 to 491520 tid 1 2024-04-26T04:32:54.501 INFO:tasks.rados.rados.0.smithi178.stdout:223: writing smithi17829308-8 from 491520 to 1081344 tid 2 2024-04-26T04:32:54.507 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:54.507 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:54.511 INFO:tasks.rados.rados.0.smithi178.stdout:223: writing smithi17829308-8 from 1081344 to 1638400 tid 3 2024-04-26T04:32:54.512 INFO:tasks.rados.rados.0.smithi178.stdout:223: writing smithi17829308-8 from 1638400 to 1654784 tid 4 2024-04-26T04:32:54.512 INFO:tasks.rados.rados.0.smithi178.stdout:224: rmattr oid 14 current snap is 12 2024-04-26T04:32:54.512 INFO:tasks.rados.rados.0.smithi178.stdout:224: done (6 left) 2024-04-26T04:32:54.512 INFO:tasks.rados.rados.0.smithi178.stdout:225: read oid 10 snap 9 2024-04-26T04:32:54.512 INFO:tasks.rados.rados.0.smithi178.stdout:225: expect (ObjNum 83 snap 9 seq_num 83) 2024-04-26T04:32:54.512 INFO:tasks.rados.rados.0.smithi178.stdout:226: read oid 37 snap -1 2024-04-26T04:32:54.512 INFO:tasks.rados.rados.0.smithi178.stdout:226: expect (ObjNum 36 snap 0 seq_num 36) 2024-04-26T04:32:54.513 INFO:tasks.rados.rados.0.smithi178.stdout:227: delete oid 48 current snap is 12 2024-04-26T04:32:54.513 INFO:tasks.rados.rados.0.smithi178.stdout:217: finishing write tid 5 to smithi17829308-44 2024-04-26T04:32:54.513 INFO:tasks.rados.rados.0.smithi178.stdout:217: oid 44 updating version 0 to 54 2024-04-26T04:32:54.513 INFO:tasks.rados.rados.0.smithi178.stdout:217: oid 44 updating version 54 to 55 2024-04-26T04:32:54.513 INFO:tasks.rados.rados.0.smithi178.stdout:217: oid 44 version 55 is already newer than 53 2024-04-26T04:32:54.513 INFO:tasks.rados.rados.0.smithi178.stdout:217: oid 44 version 55 is already newer than 52 2024-04-26T04:32:54.513 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 44 v 55 (ObjNum 95 snap 12 seq_num 95) dirty exists 2024-04-26T04:32:54.513 INFO:tasks.rados.rados.0.smithi178.stdout:217: left oid 44 (ObjNum 95 snap 12 seq_num 95) 2024-04-26T04:32:54.522 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T04:32:54.577 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:54.572+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:54.581 INFO:tasks.rados.rados.0.smithi178.stdout:215: finishing write tid 5 to smithi17829308-50 2024-04-26T04:32:54.581 INFO:tasks.rados.rados.0.smithi178.stdout:215: oid 50 updating version 0 to 25 2024-04-26T04:32:54.581 INFO:tasks.rados.rados.0.smithi178.stdout:215: oid 50 updating version 25 to 28 2024-04-26T04:32:54.581 INFO:tasks.rados.rados.0.smithi178.stdout:215: oid 50 version 28 is already newer than 26 2024-04-26T04:32:54.581 INFO:tasks.rados.rados.0.smithi178.stdout:215: oid 50 version 28 is already newer than 27 2024-04-26T04:32:54.581 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 50 v 28 (ObjNum 94 snap 12 seq_num 94) dirty exists 2024-04-26T04:32:54.581 INFO:tasks.rados.rados.0.smithi178.stdout:215: left oid 50 (ObjNum 94 snap 12 seq_num 94) 2024-04-26T04:32:54.646 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:54.646 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:54.660 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T04:32:54.678 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:54.677+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:54.690 INFO:tasks.rados.rados.0.smithi178.stdout:220: expect (ObjNum 34 snap 0 seq_num 34) 2024-04-26T04:32:54.741 INFO:tasks.rados.rados.0.smithi178.stdout:223: finishing write tid 1 to smithi17829308-8 2024-04-26T04:32:54.779 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:54.777+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:54.782 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:54.782 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:54.787 INFO:tasks.rados.rados.0.smithi178.stdout:219: expect (ObjNum 55 snap 1 seq_num 55) 2024-04-26T04:32:54.797 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T04:32:54.880 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:32:54.876+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:32:54.893 INFO:tasks.rados.rados.0.smithi178.stdout:226: expect (ObjNum 36 snap 0 seq_num 36) 2024-04-26T04:32:54.909 INFO:tasks.rados.rados.0.smithi178.stdout:223: finishing write tid 2 to smithi17829308-8 2024-04-26T04:32:54.909 INFO:tasks.rados.rados.0.smithi178.stdout:225: expect (ObjNum 83 snap 9 seq_num 83) 2024-04-26T04:32:54.960 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:54.960 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:54.971 INFO:tasks.rados.rados.0.smithi178.stdout:221: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-26T04:32:54.974 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T04:32:54.981 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:54.977+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:54.985 INFO:tasks.rados.rados.0.smithi178.stdout:223: finishing write tid 3 to smithi17829308-8 2024-04-26T04:32:54.985 INFO:tasks.rados.rados.0.smithi178.stdout:223: finishing write tid 4 to smithi17829308-8 2024-04-26T04:32:54.985 INFO:tasks.rados.rados.0.smithi178.stdout:215: done (8 left) 2024-04-26T04:32:54.985 INFO:tasks.rados.rados.0.smithi178.stdout:217: done (7 left) 2024-04-26T04:32:54.985 INFO:tasks.rados.rados.0.smithi178.stdout:219: done (6 left) 2024-04-26T04:32:54.985 INFO:tasks.rados.rados.0.smithi178.stdout:220: done (5 left) 2024-04-26T04:32:54.985 INFO:tasks.rados.rados.0.smithi178.stdout:221: done (4 left) 2024-04-26T04:32:54.985 INFO:tasks.rados.rados.0.smithi178.stdout:225: done (3 left) 2024-04-26T04:32:54.985 INFO:tasks.rados.rados.0.smithi178.stdout:226: done (2 left) 2024-04-26T04:32:54.986 INFO:tasks.rados.rados.0.smithi178.stdout:227: done (1 left) 2024-04-26T04:32:54.986 INFO:tasks.rados.rados.0.smithi178.stdout:228: snap_create 2024-04-26T04:32:54.998 INFO:tasks.rados.rados.0.smithi178.stdout:223: finishing write tid 5 to smithi17829308-8 2024-04-26T04:32:55.082 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:55.078+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:55.095 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:55.095 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:55.102 INFO:tasks.rados.rados.0.smithi178.stdout:223: finishing write tid 6 to smithi17829308-8 2024-04-26T04:32:55.102 INFO:tasks.rados.rados.0.smithi178.stdout:223: oid 8 updating version 0 to 7 2024-04-26T04:32:55.102 INFO:tasks.rados.rados.0.smithi178.stdout:223: oid 8 updating version 7 to 8 2024-04-26T04:32:55.102 INFO:tasks.rados.rados.0.smithi178.stdout:223: oid 8 updating version 8 to 9 2024-04-26T04:32:55.102 INFO:tasks.rados.rados.0.smithi178.stdout:223: oid 8 updating version 9 to 11 2024-04-26T04:32:55.102 INFO:tasks.rados.rados.0.smithi178.stdout:223: oid 8 version 11 is already newer than 10 2024-04-26T04:32:55.102 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 8 v 11 (ObjNum 98 snap 12 seq_num 98) dirty exists 2024-04-26T04:32:55.102 INFO:tasks.rados.rados.0.smithi178.stdout:223: left oid 8 (ObjNum 98 snap 12 seq_num 98) 2024-04-26T04:32:55.109 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T04:32:55.183 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:55.184+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:55.238 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:55.239 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:55.252 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T04:32:55.296 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:55.297+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:55.374 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:55.374 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:55.384 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:55.381+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:55.389 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T04:32:55.408 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.13 primary-affinity to 0.44389 (8290902) 2024-04-26T04:32:55.414 INFO:tasks.rados.rados.0.smithi178.stdout:223: done (1 left) 2024-04-26T04:32:55.414 INFO:tasks.rados.rados.0.smithi178.stdout:228: done (0 left) 2024-04-26T04:32:55.414 INFO:tasks.rados.rados.0.smithi178.stdout:229: delete oid 41 current snap is 13 2024-04-26T04:32:55.416 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T04:32:55.444 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-26T04:32:55.521 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:55.521+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:55.558 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:55.559 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:55.569 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:55.569 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:55.573 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T04:32:55.586 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:55.584+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:55.669 INFO:tasks.rados.rados.0.smithi178.stdout:229: done (0 left) 2024-04-26T04:32:55.670 INFO:tasks.rados.rados.0.smithi178.stdout:230: read oid 13 snap -1 2024-04-26T04:32:55.670 INFO:tasks.rados.rados.0.smithi178.stdout:230: expect (ObjNum 23 snap 0 seq_num 23) 2024-04-26T04:32:55.670 INFO:tasks.rados.rados.0.smithi178.stdout:231: setattr oid 26 current snap is 13 2024-04-26T04:32:55.676 INFO:tasks.rados.rados.0.smithi178.stdout:232: copy_from oid 32 from oid 7 current snap is 13 2024-04-26T04:32:55.676 INFO:tasks.rados.rados.0.smithi178.stdout:233: read oid 22 snap -1 2024-04-26T04:32:55.676 INFO:tasks.rados.rados.0.smithi178.stdout:233: expect (ObjNum 21 snap 0 seq_num 21) 2024-04-26T04:32:55.676 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 15 current snap is 13 2024-04-26T04:32:55.677 INFO:tasks.rados.rados.0.smithi178.stdout:234: seq_num 101 ranges {1458176=540672,1998848=507904,2506752=622592,3129344=32768} 2024-04-26T04:32:55.687 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:55.685+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:55.691 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:55.691 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:55.698 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:55.698 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:55.709 INFO:tasks.rados.rados.0.smithi178.stdout:234: writing smithi17829308-15 from 1458176 to 1998848 tid 1 2024-04-26T04:32:55.712 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T04:32:55.717 INFO:tasks.rados.rados.0.smithi178.stdout:234: writing smithi17829308-15 from 1998848 to 2506752 tid 2 2024-04-26T04:32:55.722 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:55.722 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:55.726 INFO:tasks.rados.rados.0.smithi178.stdout:234: writing smithi17829308-15 from 2506752 to 3129344 tid 3 2024-04-26T04:32:55.726 INFO:tasks.rados.rados.0.smithi178.stdout:234: writing smithi17829308-15 from 3129344 to 3162112 tid 4 2024-04-26T04:32:55.726 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 49 current snap is 13 2024-04-26T04:32:55.726 INFO:tasks.rados.rados.0.smithi178.stdout:235: seq_num 102 ranges {1949696=557056,2506752=638976,3145728=524288} 2024-04-26T04:32:55.751 INFO:tasks.rados.rados.0.smithi178.stdout:235: writing smithi17829308-49 from 1949696 to 2506752 tid 1 2024-04-26T04:32:55.753 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:55.754 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:55.756 INFO:tasks.rados.rados.0.smithi178.stdout:235: writing smithi17829308-49 from 2506752 to 3145728 tid 2 2024-04-26T04:32:55.764 INFO:tasks.rados.rados.0.smithi178.stdout:235: writing smithi17829308-49 from 3145728 to 3670016 tid 3 2024-04-26T04:32:55.764 INFO:tasks.rados.rados.0.smithi178.stdout:236: rollback oid 33 current snap is 13 2024-04-26T04:32:55.764 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 33 to 9 2024-04-26T04:32:55.764 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 30 current snap is 13 2024-04-26T04:32:55.764 INFO:tasks.rados.rados.0.smithi178.stdout:237: seq_num 103 ranges {491520=573440,1064960=786432,1851392=376832} 2024-04-26T04:32:55.768 INFO:tasks.rados.rados.0.smithi178.stdout:237: writing smithi17829308-30 from 491520 to 1064960 tid 1 2024-04-26T04:32:55.774 INFO:tasks.rados.rados.0.smithi178.stdout:237: writing smithi17829308-30 from 1064960 to 1851392 tid 2 2024-04-26T04:32:55.777 INFO:tasks.rados.rados.0.smithi178.stdout:237: writing smithi17829308-30 from 1851392 to 2228224 tid 3 2024-04-26T04:32:55.777 INFO:tasks.rados.rados.0.smithi178.stdout:238: read oid 37 snap -1 2024-04-26T04:32:55.777 INFO:tasks.rados.rados.0.smithi178.stdout:238: expect (ObjNum 36 snap 0 seq_num 36) 2024-04-26T04:32:55.777 INFO:tasks.rados.rados.0.smithi178.stdout:239: rollback oid 8 current snap is 13 2024-04-26T04:32:55.777 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 8 to 12 2024-04-26T04:32:55.777 INFO:tasks.rados.rados.0.smithi178.stdout:240: rmattr oid 46 current snap is 13 2024-04-26T04:32:55.777 INFO:tasks.rados.rados.0.smithi178.stdout:241: copy_from oid 10 from oid 6 current snap is 13 2024-04-26T04:32:55.777 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 16 current snap is 13 2024-04-26T04:32:55.777 INFO:tasks.rados.rados.0.smithi178.stdout:242: seq_num 104 ranges {0=589824,589824=540672,1130496=622592} 2024-04-26T04:32:55.780 INFO:tasks.rados.rados.0.smithi178.stdout:242: writing smithi17829308-16 from 0 to 589824 tid 1 2024-04-26T04:32:55.782 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:55.783 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:55.783 INFO:tasks.rados.rados.0.smithi178.stdout:242: writing smithi17829308-16 from 589824 to 1130496 tid 2 2024-04-26T04:32:55.787 INFO:tasks.rados.rados.0.smithi178.stdout:242: writing smithi17829308-16 from 1130496 to 1753088 tid 3 2024-04-26T04:32:55.787 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 40 current snap is 13 2024-04-26T04:32:55.787 INFO:tasks.rados.rados.0.smithi178.stdout:243: seq_num 105 ranges {1622016=606208,2228224=688128,2916352=475136} 2024-04-26T04:32:55.788 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:55.786+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:55.800 INFO:tasks.rados.rados.0.smithi178.stdout:243: writing smithi17829308-40 from 1622016 to 2228224 tid 1 2024-04-26T04:32:55.803 INFO:tasks.rados.rados.0.smithi178.stdout:243: writing smithi17829308-40 from 2228224 to 2916352 tid 2 2024-04-26T04:32:55.806 INFO:tasks.rados.rados.0.smithi178.stdout:243: writing smithi17829308-40 from 2916352 to 3391488 tid 3 2024-04-26T04:32:55.806 INFO:tasks.rados.rados.0.smithi178.stdout:244: read oid 14 snap 9 2024-04-26T04:32:55.807 INFO:tasks.rados.rados.0.smithi178.stdout:244: expect (ObjNum 13 snap 0 seq_num 13) 2024-04-26T04:32:55.807 INFO:tasks.rados.rados.0.smithi178.stdout:245: rmattr oid 21 current snap is 13 2024-04-26T04:32:55.807 INFO:tasks.rados.rados.0.smithi178.stdout:245: done (15 left) 2024-04-26T04:32:55.807 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 35 current snap is 13 2024-04-26T04:32:55.807 INFO:tasks.rados.rados.0.smithi178.stdout:246: seq_num 106 ranges {573440=622592,1196032=442368,1638400=524288,2162688=196608} 2024-04-26T04:32:55.809 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:55.809 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:55.813 INFO:tasks.rados.rados.0.smithi178.stdout:246: writing smithi17829308-35 from 573440 to 1196032 tid 1 2024-04-26T04:32:55.815 INFO:tasks.rados.rados.0.smithi178.stdout:246: writing smithi17829308-35 from 1196032 to 1638400 tid 2 2024-04-26T04:32:55.818 INFO:tasks.rados.rados.0.smithi178.stdout:246: writing smithi17829308-35 from 1638400 to 2162688 tid 3 2024-04-26T04:32:55.820 INFO:tasks.rados.rados.0.smithi178.stdout:246: writing smithi17829308-35 from 2162688 to 2359296 tid 4 2024-04-26T04:32:55.820 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:55.832 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:55.833 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:55.839 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:55.839 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:55.853 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T04:32:55.888 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:55.889+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:55.905 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:55.905 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:55.982 INFO:tasks.rados.rados.0.smithi178.stdout:230: expect (ObjNum 23 snap 0 seq_num 23) 2024-04-26T04:32:55.986 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:55.986 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:55.986 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:32:55.986 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:55.990 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:55.989+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:55.993 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 26 v 50 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-04-26T04:32:55.993 INFO:tasks.rados.rados.0.smithi178.stdout:230: done (15 left) 2024-04-26T04:32:55.993 INFO:tasks.rados.rados.0.smithi178.stdout:231: done (14 left) 2024-04-26T04:32:55.993 INFO:tasks.rados.rados.0.smithi178.stdout:247: read oid 39 snap -1 2024-04-26T04:32:55.993 INFO:tasks.rados.rados.0.smithi178.stdout:247: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:32:55.993 INFO:tasks.rados.rados.0.smithi178.stdout:248: read oid 28 snap 9 2024-04-26T04:32:55.993 INFO:tasks.rados.rados.0.smithi178.stdout:248: expect (ObjNum 27 snap 0 seq_num 27) 2024-04-26T04:32:55.993 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:56.001 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T04:32:56.020 INFO:tasks.rados.rados.0.smithi178.stdout:234: finishing write tid 1 to smithi17829308-15 2024-04-26T04:32:56.085 INFO:tasks.rados.rados.0.smithi178.stdout:236: finishing rollback tid 0 to smithi17829308-33 2024-04-26T04:32:56.085 INFO:tasks.rados.rados.0.smithi178.stdout:236: finishing rollback tid 1 to smithi17829308-33 2024-04-26T04:32:56.087 INFO:tasks.rados.rados.0.smithi178.stdout:237: finishing write tid 1 to smithi17829308-30 2024-04-26T04:32:56.087 INFO:tasks.rados.rados.0.smithi178.stdout:236: finishing rollback tid 2 to smithi17829308-33 2024-04-26T04:32:56.087 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 33 v 32 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-04-26T04:32:56.087 INFO:tasks.rados.rados.0.smithi178.stdout:236: done (15 left) 2024-04-26T04:32:56.087 INFO:tasks.rados.rados.0.smithi178.stdout:249: setattr oid 43 current snap is 13 2024-04-26T04:32:56.088 INFO:tasks.rados.rados.0.smithi178.stdout:242: finishing write tid 1 to smithi17829308-16 2024-04-26T04:32:56.088 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:56.089 INFO:tasks.rados.rados.0.smithi178.stdout:242: finishing write tid 2 to smithi17829308-16 2024-04-26T04:32:56.091 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:56.089+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:56.095 INFO:tasks.rados.rados.0.smithi178.stdout:241: finishing copy_from to smithi17829308-10 2024-04-26T04:32:56.095 INFO:tasks.rados.rados.0.smithi178.stdout:241: got expected ENOENT (src dne) 2024-04-26T04:32:56.095 INFO:tasks.rados.rados.0.smithi178.stdout:241: finishing copy_from racing read to smithi17829308-10 2024-04-26T04:32:56.096 INFO:tasks.rados.rados.0.smithi178.stdout:241: done (15 left) 2024-04-26T04:32:56.096 INFO:tasks.rados.rados.0.smithi178.stdout:250: read oid 2 snap -1 2024-04-26T04:32:56.096 INFO:tasks.rados.rados.0.smithi178.stdout:250: expect (ObjNum 88 snap 12 seq_num 88) 2024-04-26T04:32:56.096 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 16 2024-04-26T04:32:56.129 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:56.129 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:56.129 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:32:56.129 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:32:56.129 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:56.143 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T04:32:56.165 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-04-26T04:32:56.179 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 46 v 56 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-26T04:32:56.180 INFO:tasks.rados.rados.0.smithi178.stdout:240: done (15 left) 2024-04-26T04:32:56.180 INFO:tasks.rados.rados.0.smithi178.stdout:251: snap_create 2024-04-26T04:32:56.183 INFO:tasks.rados.rados.0.smithi178.stdout:235: finishing write tid 1 to smithi17829308-49 2024-04-26T04:32:56.183 INFO:tasks.rados.rados.0.smithi178.stdout:235: finishing write tid 2 to smithi17829308-49 2024-04-26T04:32:56.183 INFO:tasks.rados.rados.0.smithi178.stdout:235: finishing write tid 3 to smithi17829308-49 2024-04-26T04:32:56.192 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:56.190+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:56.276 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:32:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:32:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:32:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:42 1.0 e26/23 1:30306672:devicehealth::main.db.0000000000000000:head v 26'28, mlcod=26'26)", 2024-04-26T04:32:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:32:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 43.054177154999998, 2024-04-26T04:32:56.277 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201587136, 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.278 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220676+0000", 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.843e-06 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220687+0000", 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1727e-05 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220715+0000", 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8033000000000001e-05 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.279 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.320857+0000", 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014171400000001 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.421143+0000", 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100285902 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.421175+0000", 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2413999999999998e-05 2024-04-26T04:32:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.422168+0000", 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0009931269999999999 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.422261+0000", 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2375999999999998e-05 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.281 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:55 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'33, mlcod=27'29)", 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 42.730787292000002, 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.169473288, 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.282 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544065+0000", 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.77e-06 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544195+0000", 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012971599999999999 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.283 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.284 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544219+0000", 2024-04-26T04:32:56.284 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3476e-05 2024-04-26T04:32:56.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.284 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.284 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.284 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.644316+0000", 2024-04-26T04:32:56.284 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100097189 2024-04-26T04:32:56.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.284 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.284 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.284 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.711737+0000", 2024-04-26T04:32:56.284 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.067420846000000006 2024-04-26T04:32:56.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.711770+0000", 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2959000000000001e-05 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.713428+0000", 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016584130000000001 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.713537+0000", 2024-04-26T04:32:56.285 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010891900000000001 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:108 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'70, mlcod=27'64)", 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 39.466393128999997, 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30260933299999998, 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.286 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.37e-07 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.287 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808464+0000", 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8599999999999998e-06 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808474+0000", 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.9830000000000008e-06 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908648+0000", 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10017390299999999 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.288 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908808+0000", 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000159576 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908845+0000", 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7026000000000001e-05 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110983+0000", 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20213835199999999 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.289 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111067+0000", 2024-04-26T04:32:56.290 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3996e-05 2024-04-26T04:32:56.290 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.290 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.290 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.290 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.290 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.290 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:109 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'71, mlcod=27'64)", 2024-04-26T04:32:56.290 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:32:56.290 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 39.366158405999997, 2024-04-26T04:32:56.290 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202472819, 2024-04-26T04:32:56.290 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.290 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.290 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.291 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908694+0000", 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.144e-06 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908751+0000", 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7268999999999997e-05 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908769+0000", 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8363e-05 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.292 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.293 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908780+0000", 2024-04-26T04:32:56.293 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.096e-05 2024-04-26T04:32:56.293 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.293 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.293 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.293 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.909341+0000", 2024-04-26T04:32:56.293 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00056057500000000001 2024-04-26T04:32:56.293 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.293 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.293 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.293 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.909371+0000", 2024-04-26T04:32:56.293 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0357000000000001e-05 2024-04-26T04:32:56.293 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111074+0000", 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20170292500000001 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111165+0000", 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.1225999999999997e-05 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.294 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.295 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:110 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'72, mlcod=27'64)", 2024-04-26T04:32:56.295 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:32:56.295 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 39.366023513999998, 2024-04-26T04:32:56.295 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20385624999999999, 2024-04-26T04:32:56.295 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.295 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.295 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.295 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.295 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.295 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.295 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:32:56.295 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908828+0000", 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9400000000000005e-07 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.296 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908836+0000", 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1929999999999993e-06 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908847+0000", 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0703000000000001e-05 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908855+0000", 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7829999999999999e-06 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.297 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009054+0000", 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10019966900000001 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009089+0000", 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4433e-05 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112582+0000", 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103492973 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.298 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112684+0000", 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000101802 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:111 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'73, mlcod=27'64)", 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 39.365984584000003, 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.205422306, 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.299 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.300 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908867+0000", 2024-04-26T04:32:56.301 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.03e-07 2024-04-26T04:32:56.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.301 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.301 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.301 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908871+0000", 2024-04-26T04:32:56.301 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5800000000000002e-06 2024-04-26T04:32:56.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.301 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.301 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.301 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908883+0000", 2024-04-26T04:32:56.301 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1507e-05 2024-04-26T04:32:56.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.301 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.302 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.302 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009022+0000", 2024-04-26T04:32:56.302 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100139593 2024-04-26T04:32:56.302 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.302 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.302 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.302 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.109703+0000", 2024-04-26T04:32:56.302 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100680741 2024-04-26T04:32:56.302 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.302 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.302 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.302 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.109738+0000", 2024-04-26T04:32:56.302 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5296000000000002e-05 2024-04-26T04:32:56.303 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.303 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.303 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.303 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114207+0000", 2024-04-26T04:32:56.303 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0044682239999999998 2024-04-26T04:32:56.303 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.303 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.303 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.303 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114289+0000", 2024-04-26T04:32:56.303 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2062000000000001e-05 2024-04-26T04:32:56.303 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.303 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.303 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:112 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'74, mlcod=27'64)", 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 39.265790461999998, 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105317487, 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:32:56.304 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:32:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009061+0000", 2024-04-26T04:32:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.057e-06 2024-04-26T04:32:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.306 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009128+0000", 2024-04-26T04:32:56.306 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6859999999999993e-05 2024-04-26T04:32:56.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.306 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.306 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.306 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009147+0000", 2024-04-26T04:32:56.306 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8196e-05 2024-04-26T04:32:56.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.306 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.306 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.306 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009157+0000", 2024-04-26T04:32:56.306 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0743e-05 2024-04-26T04:32:56.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110254+0000", 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101096941 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110311+0000", 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6718999999999997e-05 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114296+0000", 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0039847709999999998 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.307 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114378+0000", 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2200000000000006e-05 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:114 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'76, mlcod=27'64)", 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 39.265641213000002, 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105329463, 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.308 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:32:56.309 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.53e-07 2024-04-26T04:32:56.310 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.310 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.310 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.310 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009215+0000", 2024-04-26T04:32:56.310 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7099999999999998e-06 2024-04-26T04:32:56.310 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.310 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.310 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.310 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009223+0000", 2024-04-26T04:32:56.310 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5599999999999994e-06 2024-04-26T04:32:56.310 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.310 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.310 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009229+0000", 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0800000000000002e-06 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111865+0000", 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102636071 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111941+0000", 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5615e-05 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.311 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114468+0000", 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002526875 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114539+0000", 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1198999999999994e-05 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:115 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'77, mlcod=27'64)", 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:32:56.312 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 39.265613340000002, 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105390922, 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:32:56.313 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:32:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.34e-07 2024-04-26T04:32:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009293+0000", 2024-04-26T04:32:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4954999999999998e-05 2024-04-26T04:32:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.314 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009304+0000", 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1263999999999999e-05 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009311+0000", 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6630000000000004e-06 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112802+0000", 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10349130600000001 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.315 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112843+0000", 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0472999999999997e-05 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114547+0000", 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001704552 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114628+0000", 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1374999999999999e-05 2024-04-26T04:32:56.316 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.317 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.317 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.317 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.317 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.317 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:116 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'78, mlcod=27'64)", 2024-04-26T04:32:56.317 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:32:56.317 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 39.265530087000002, 2024-04-26T04:32:56.317 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105877241, 2024-04-26T04:32:56.317 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.317 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.317 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.317 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.317 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.318 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.318 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:32:56.318 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.318 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.318 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.318 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:32:56.318 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.318 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.318 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.318 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:32:56.318 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.46e-07 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009326+0000", 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5040000000000004e-06 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009334+0000", 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1929999999999993e-06 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.319 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009340+0000", 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.3729999999999998e-06 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.113595+0000", 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104254889 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.113649+0000", 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3887000000000001e-05 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.320 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.321 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115105+0000", 2024-04-26T04:32:56.321 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014560969999999999 2024-04-26T04:32:56.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.321 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.321 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.321 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115198+0000", 2024-04-26T04:32:56.321 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2851999999999998e-05 2024-04-26T04:32:56.321 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.321 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.321 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.321 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.321 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:117 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'79, mlcod=27'64)", 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 39.265502603999998, 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10639752, 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.322 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009349+0000", 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7300000000000002e-07 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009353+0000", 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5970000000000002e-06 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.323 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009361+0000", 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0600000000000008e-06 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009381+0000", 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9533000000000001e-05 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114709+0000", 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105328753 2024-04-26T04:32:56.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114745+0000", 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5976000000000002e-05 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115683+0000", 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00093774899999999998 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115746+0000", 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2578999999999998e-05 2024-04-26T04:32:56.325 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.326 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.326 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.326 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.326 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:155 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'109, mlcod=27'105)", 2024-04-26T04:32:56.326 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:32:56.326 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 37.435314830999999, 2024-04-26T04:32:56.326 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20562082700000001, 2024-04-26T04:32:56.326 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.326 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.326 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.326 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.326 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839537+0000", 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7400000000000003e-07 2024-04-26T04:32:56.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839638+0000", 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000101365 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839653+0000", 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4874e-05 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939797+0000", 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014414100000001 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.328 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.040092+0000", 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100295305 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.040151+0000", 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8295999999999999e-05 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045057+0000", 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0049061390000000003 2024-04-26T04:32:56.329 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045157+0000", 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000100133 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:156 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'110, mlcod=27'105)", 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 37.335019354000003, 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10750937100000001, 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.330 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.331 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939833+0000", 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1030000000000001e-06 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939849+0000", 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6005999999999999e-05 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939866+0000", 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7513e-05 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939876+0000", 2024-04-26T04:32:56.332 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0154e-05 2024-04-26T04:32:56.333 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.333 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.333 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.333 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.041330+0000", 2024-04-26T04:32:56.333 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101453864 2024-04-26T04:32:56.333 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.333 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.333 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.333 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.041398+0000", 2024-04-26T04:32:56.333 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7617e-05 2024-04-26T04:32:56.333 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.333 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.333 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.334 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047245+0000", 2024-04-26T04:32:56.334 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058470140000000002 2024-04-26T04:32:56.334 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.334 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.334 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.334 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047341+0000", 2024-04-26T04:32:56.334 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6100000000000005e-05 2024-04-26T04:32:56.334 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.334 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.334 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.334 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.334 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.334 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:157 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'111, mlcod=27'105)", 2024-04-26T04:32:56.335 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:32:56.335 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 37.334919214999999, 2024-04-26T04:32:56.335 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107487887, 2024-04-26T04:32:56.335 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.335 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.335 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.335 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.335 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.335 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.335 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:32:56.335 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.335 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2300000000000001e-07 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939942+0000", 2024-04-26T04:32:56.336 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4369999999999996e-06 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939952+0000", 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.075e-05 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939963+0000", 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0069e-05 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.042203+0000", 2024-04-26T04:32:56.337 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102240215 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.042259+0000", 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5955999999999998e-05 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047353+0000", 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0050939080000000003 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.338 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047420+0000", 2024-04-26T04:32:56.339 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6928999999999996e-05 2024-04-26T04:32:56.339 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.339 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.339 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.339 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.339 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.339 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:158 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'112, mlcod=27'105)", 2024-04-26T04:32:56.339 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:32:56.339 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 37.334876629, 2024-04-26T04:32:56.339 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.209867842, 2024-04-26T04:32:56.339 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.339 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.340 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939975+0000", 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8599999999999999e-07 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940084+0000", 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000109746 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940099+0000", 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4392999999999999e-05 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.341 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940110+0000", 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1236999999999999e-05 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043002+0000", 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102892262 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043054+0000", 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1922999999999997e-05 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.342 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149774+0000", 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10672018599999999 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149842+0000", 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7708999999999998e-05 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:159 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'113, mlcod=27'105)", 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 37.334728007000002, 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20976125000000001, 2024-04-26T04:32:56.343 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.344 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7000000000000005e-07 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940131+0000", 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6369999999999999e-06 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940143+0000", 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1603000000000001e-05 2024-04-26T04:32:56.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940165+0000", 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2042999999999999e-05 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043777+0000", 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103612145 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043807+0000", 2024-04-26T04:32:56.346 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0562999999999999e-05 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149845+0000", 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10603802900000001 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149884+0000", 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8659999999999999e-05 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.347 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:160 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'114, mlcod=27'105)", 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 37.334677501000002, 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20974973299999999, 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:32:56.348 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940174+0000", 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8999999999999998e-07 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940183+0000", 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6970000000000005e-06 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940194+0000", 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0983e-05 2024-04-26T04:32:56.349 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940201+0000", 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4400000000000002e-06 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.044326+0000", 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104124984 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.044377+0000", 2024-04-26T04:32:56.350 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1249999999999999e-05 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149886+0000", 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10550875999999999 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149923+0000", 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7329000000000001e-05 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.351 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.352 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:161 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'115, mlcod=27'105)", 2024-04-26T04:32:56.352 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:32:56.352 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 37.334641558000001, 2024-04-26T04:32:56.352 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.209755314, 2024-04-26T04:32:56.352 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.352 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.352 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.352 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.352 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.352 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.352 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:32:56.352 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.352 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940210+0000", 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.67e-07 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940330+0000", 2024-04-26T04:32:56.353 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012061900000000001 2024-04-26T04:32:56.354 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.354 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.354 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.354 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940344+0000", 2024-04-26T04:32:56.354 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.367e-05 2024-04-26T04:32:56.354 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.354 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.354 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.354 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940355+0000", 2024-04-26T04:32:56.354 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.083e-05 2024-04-26T04:32:56.354 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.354 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045239+0000", 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104883873 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045278+0000", 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9904000000000003e-05 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149925+0000", 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104646221 2024-04-26T04:32:56.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149965+0000", 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9929999999999999e-05 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:162 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'116, mlcod=27'105)", 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 37.331177826000001, 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.206336451, 2024-04-26T04:32:56.356 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.357 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943675+0000", 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8640000000000001e-06 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943692+0000", 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7503000000000002e-05 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943720+0000", 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7569999999999999e-05 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.358 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943734+0000", 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3596e-05 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.046517+0000", 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102783897 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.046569+0000", 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1706000000000003e-05 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.359 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149968+0000", 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10339902199999999 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150009+0000", 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1292999999999999e-05 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:163 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'117, mlcod=27'105)", 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:32:56.360 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 37.329676755999998, 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20488266699999999, 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:32:56.361 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945176+0000", 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8470000000000001e-06 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945193+0000", 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7013000000000001e-05 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.362 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945221+0000", 2024-04-26T04:32:56.363 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7670000000000001e-05 2024-04-26T04:32:56.363 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.363 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.363 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.363 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945238+0000", 2024-04-26T04:32:56.363 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7113e-05 2024-04-26T04:32:56.363 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.363 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.363 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.363 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.147614+0000", 2024-04-26T04:32:56.363 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202375893 2024-04-26T04:32:56.363 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.363 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.364 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:56.364 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.147669+0000", 2024-04-26T04:32:56.364 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5729000000000001e-05 2024-04-26T04:32:56.364 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.364 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.364 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:56.364 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150011+0000", 2024-04-26T04:32:56.364 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002342083 2024-04-26T04:32:56.364 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.364 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.364 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.364 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150057+0000", 2024-04-26T04:32:56.364 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5318999999999998e-05 2024-04-26T04:32:56.365 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.365 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.365 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.365 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.365 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.365 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:56.366 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:56.367 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:56.367 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:56.368 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:56.368 INFO:tasks.rados.rados.0.smithi178.stdout:234: finishing write tid 2 to smithi17829308-15 2024-04-26T04:32:56.368 INFO:tasks.rados.rados.0.smithi178.stdout:234: finishing write tid 3 to smithi17829308-15 2024-04-26T04:32:56.368 INFO:tasks.rados.rados.0.smithi178.stdout:235: finishing write tid 4 to smithi17829308-49 2024-04-26T04:32:56.368 INFO:tasks.rados.rados.0.smithi178.stdout:232: finishing copy_from racing read to smithi17829308-32 2024-04-26T04:32:56.368 INFO:tasks.rados.rados.0.smithi178.stdout:243: finishing write tid 1 to smithi17829308-40 2024-04-26T04:32:56.368 INFO:tasks.rados.rados.0.smithi178.stdout:243: finishing write tid 2 to smithi17829308-40 2024-04-26T04:32:56.369 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T04:32:56.369 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:56.370 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:56.370 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:56.365+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:56.374 INFO:tasks.rados.rados.0.smithi178.stdout:239: finishing rollback tid 0 to smithi17829308-8 2024-04-26T04:32:56.379 INFO:tasks.rados.rados.0.smithi178.stdout:234: finishing write tid 4 to smithi17829308-15 2024-04-26T04:32:56.383 INFO:tasks.rados.rados.0.smithi178.stdout:234: finishing write tid 5 to smithi17829308-15 2024-04-26T04:32:56.387 INFO:tasks.rados.rados.0.smithi178.stdout:248: expect (ObjNum 27 snap 0 seq_num 27) 2024-04-26T04:32:56.394 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:56.394 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:56.420 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:56.420 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:56.432 INFO:tasks.rados.rados.0.smithi178.stdout:243: finishing write tid 3 to smithi17829308-40 2024-04-26T04:32:56.432 INFO:tasks.rados.rados.0.smithi178.stdout:243: finishing write tid 4 to smithi17829308-40 2024-04-26T04:32:56.467 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:56.465+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:56.471 INFO:tasks.rados.rados.0.smithi178.stdout:239: finishing rollback tid 1 to smithi17829308-8 2024-04-26T04:32:56.471 INFO:tasks.rados.rados.0.smithi178.stdout:239: finishing rollback tid 2 to smithi17829308-8 2024-04-26T04:32:56.471 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 8 v 15 (ObjNum 98 snap 12 seq_num 98) dirty exists 2024-04-26T04:32:56.487 INFO:tasks.rados.rados.0.smithi178.stdout:237: finishing write tid 2 to smithi17829308-30 2024-04-26T04:32:56.497 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:56.497 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:56.497 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:32:56.498 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:56.501 INFO:tasks.rados.rados.0.smithi178.stdout:242: finishing write tid 3 to smithi17829308-16 2024-04-26T04:32:56.501 INFO:tasks.rados.rados.0.smithi178.stdout:242: finishing write tid 4 to smithi17829308-16 2024-04-26T04:32:56.513 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T04:32:56.517 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 43 v 38 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-04-26T04:32:56.519 INFO:tasks.rados.rados.0.smithi178.stdout:250: expect (ObjNum 88 snap 12 seq_num 88) 2024-04-26T04:32:56.523 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:56.524 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:56.551 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:56.551 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:56.567 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:56.565+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:56.580 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:56.580 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:56.636 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:56.636 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:56.636 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:32:56.636 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:32:56.636 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:56.651 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T04:32:56.668 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:56.666+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:56.769 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:56.766+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:56.772 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:56.773 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:32:56.773 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:32:56.773 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:32:56.773 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.773 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 32/30 ECSubWrite(tid=0, reqid=client.4670.0:1, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:56.773 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.202677+0000", 2024-04-26T04:32:56.773 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 31.569042506999999, 2024-04-26T04:32:56.773 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19503779500000001, 2024-04-26T04:32:56.773 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.773 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.773 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.773 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.774 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.774 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.774 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202677+0000", 2024-04-26T04:32:56.774 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.774 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.774 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.774 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.774 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202677+0000", 2024-04-26T04:32:56.774 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.774 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.775 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.775 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.775 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202678+0000", 2024-04-26T04:32:56.775 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8599999999999998e-07 2024-04-26T04:32:56.775 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.775 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.775 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.775 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202736+0000", 2024-04-26T04:32:56.775 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7933000000000003e-05 2024-04-26T04:32:56.775 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.775 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.775 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.775 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202761+0000", 2024-04-26T04:32:56.776 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4706999999999999e-05 2024-04-26T04:32:56.776 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.776 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.776 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.776 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202786+0000", 2024-04-26T04:32:56.776 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5528999999999999e-05 2024-04-26T04:32:56.776 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.776 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.776 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.776 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.304898+0000", 2024-04-26T04:32:56.776 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102111946 2024-04-26T04:32:56.776 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.304915+0000", 2024-04-26T04:32:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6846000000000001e-05 2024-04-26T04:32:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397689+0000", 2024-04-26T04:32:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.092774051999999996 2024-04-26T04:32:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.777 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397715+0000", 2024-04-26T04:32:56.778 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6196000000000001e-05 2024-04-26T04:32:56.778 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.778 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.778 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.778 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.778 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.778 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 32/30 ECSubWrite(tid=1, reqid=client.4670.0:2, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:56.778 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.204544+0000", 2024-04-26T04:32:56.778 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 31.567175516999999, 2024-04-26T04:32:56.778 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19320447099999999, 2024-04-26T04:32:56.778 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.778 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.778 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.779 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.779 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.779 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.779 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204544+0000", 2024-04-26T04:32:56.779 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.779 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.779 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.779 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204544+0000", 2024-04-26T04:32:56.779 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.779 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.779 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.779 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.780 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204546+0000", 2024-04-26T04:32:56.780 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.336e-06 2024-04-26T04:32:56.780 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.780 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.780 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.780 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204575+0000", 2024-04-26T04:32:56.780 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9689999999999999e-05 2024-04-26T04:32:56.780 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.780 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.780 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.780 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204597+0000", 2024-04-26T04:32:56.781 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1593000000000001e-05 2024-04-26T04:32:56.781 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.781 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.781 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.781 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.304678+0000", 2024-04-26T04:32:56.781 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10008115400000001 2024-04-26T04:32:56.781 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.781 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.781 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.781 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305492+0000", 2024-04-26T04:32:56.781 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000813666 2024-04-26T04:32:56.781 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.782 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.782 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.782 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305505+0000", 2024-04-26T04:32:56.782 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3169999999999999e-05 2024-04-26T04:32:56.782 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.782 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.782 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.782 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397718+0000", 2024-04-26T04:32:56.782 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.092213081000000002 2024-04-26T04:32:56.782 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.786 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397749+0000", 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0781000000000002e-05 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.0s5 32/30 ECSubWrite(tid=0, reqid=client.4670.0:22, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.207578+0000", 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 31.564142228000001, 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19009067299999999, 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.787 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207578+0000", 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207578+0000", 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207579+0000", 2024-04-26T04:32:56.788 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6270000000000001e-06 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207691+0000", 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000111382 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207728+0000", 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6946999999999997e-05 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207755+0000", 2024-04-26T04:32:56.789 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7472999999999999e-05 2024-04-26T04:32:56.790 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.790 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.790 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.790 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.297516+0000", 2024-04-26T04:32:56.790 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.089760928000000004 2024-04-26T04:32:56.790 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.790 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.790 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.790 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.297533+0000", 2024-04-26T04:32:56.790 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7249999999999999e-05 2024-04-26T04:32:56.790 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.790 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.790 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.791 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397639+0000", 2024-04-26T04:32:56.791 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100105334 2024-04-26T04:32:56.791 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.791 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.791 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.791 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397668+0000", 2024-04-26T04:32:56.791 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9731999999999999e-05 2024-04-26T04:32:56.791 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.791 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.791 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.791 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.791 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.791 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.0s5 32/30 ECSubWrite(tid=1, reqid=client.4670.0:23, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:56.792 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.208255+0000", 2024-04-26T04:32:56.792 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 31.563464694, 2024-04-26T04:32:56.792 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.189431346, 2024-04-26T04:32:56.792 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.792 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.792 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.792 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.792 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.792 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.792 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208255+0000", 2024-04-26T04:32:56.792 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.792 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.793 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.793 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.793 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208255+0000", 2024-04-26T04:32:56.793 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.793 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.793 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.793 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.793 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208257+0000", 2024-04-26T04:32:56.793 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4869999999999999e-06 2024-04-26T04:32:56.793 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.793 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.793 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.793 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208294+0000", 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7679999999999998e-05 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208325+0000", 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0533000000000001e-05 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208348+0000", 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2566999999999999e-05 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.794 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.795 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.298496+0000", 2024-04-26T04:32:56.795 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.090148454000000003 2024-04-26T04:32:56.795 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.795 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.795 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.795 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.298518+0000", 2024-04-26T04:32:56.795 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2234000000000001e-05 2024-04-26T04:32:56.795 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.795 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.795 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.795 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397672+0000", 2024-04-26T04:32:56.795 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.099153924000000004 2024-04-26T04:32:56.795 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.796 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.796 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.796 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397687+0000", 2024-04-26T04:32:56.796 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4467000000000001e-05 2024-04-26T04:32:56.796 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.796 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.796 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.796 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.796 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.796 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 36/30 ECSubWrite(tid=39, reqid=client.4670.0:247, at_version=36'25, trim_to=0'0, roll_forward_to=35'24))", 2024-04-26T04:32:56.796 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.074094+0000", 2024-04-26T04:32:56.796 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.697625744, 2024-04-26T04:32:56.796 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20298102000000001, 2024-04-26T04:32:56.797 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.797 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.797 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.797 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.797 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.797 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.797 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074094+0000", 2024-04-26T04:32:56.797 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.797 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.797 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.797 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.797 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074094+0000", 2024-04-26T04:32:56.797 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.798 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.798 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.798 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.798 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074096+0000", 2024-04-26T04:32:56.798 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.854e-06 2024-04-26T04:32:56.798 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.798 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.798 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.798 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074602+0000", 2024-04-26T04:32:56.798 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00050626899999999995 2024-04-26T04:32:56.798 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.798 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.798 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.799 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074642+0000", 2024-04-26T04:32:56.799 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0113e-05 2024-04-26T04:32:56.799 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.799 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.799 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.799 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174816+0000", 2024-04-26T04:32:56.799 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10017388200000001 2024-04-26T04:32:56.799 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.799 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.799 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.799 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174981+0000", 2024-04-26T04:32:56.799 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016466500000000001 2024-04-26T04:32:56.799 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.800 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.800 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.800 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174995+0000", 2024-04-26T04:32:56.800 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.419e-05 2024-04-26T04:32:56.800 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.800 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.800 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.800 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.277024+0000", 2024-04-26T04:32:56.800 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102028498 2024-04-26T04:32:56.800 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.800 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.800 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.277075+0000", 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1548999999999997e-05 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 40/30 ECSubWrite(tid=46, reqid=osd.5.0:46, at_version=40'31, trim_to=0'0, roll_forward_to=39'29))", 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.419586+0000", 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 22.352133533, 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10595267, 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.801 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.802 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.802 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.802 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.802 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419586+0000", 2024-04-26T04:32:56.802 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.802 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.802 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.802 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.802 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419586+0000", 2024-04-26T04:32:56.802 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.802 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.802 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419587+0000", 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5700000000000002e-07 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419629+0000", 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1628999999999997e-05 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419654+0000", 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5466999999999999e-05 2024-04-26T04:32:56.803 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419670+0000", 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5699999999999999e-05 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419799+0000", 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000129085 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419813+0000", 2024-04-26T04:32:56.804 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4387e-05 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.525472+0000", 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105658988 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.525539+0000", 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6656999999999996e-05 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.805 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.as5 40/30 ECSubWrite(tid=28, reqid=client.4670.0:330, at_version=40'20, trim_to=0'0, roll_forward_to=38'16))", 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.518282+0000", 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 22.253437976000001, 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103325785, 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518282+0000", 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.806 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518282+0000", 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518284+0000", 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2869999999999999e-06 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518328+0000", 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3532000000000003e-05 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.807 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518375+0000", 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7194000000000002e-05 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.518400+0000", 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5466e-05 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.618615+0000", 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10021411600000001 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.808 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.618643+0000", 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7956000000000001e-05 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.621518+0000", 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002875945 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.621608+0000", 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.9289000000000005e-05 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.810 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.810 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.810 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.810 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 40/30 ECSubWrite(tid=104, reqid=client.4670.0:321, at_version=40'18, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:32:56.810 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.611130+0000", 2024-04-26T04:32:56.810 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 22.160589855000001, 2024-04-26T04:32:56.810 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201625046, 2024-04-26T04:32:56.810 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.810 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.810 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.810 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.810 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.810 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611130+0000", 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611130+0000", 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611131+0000", 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1300000000000001e-07 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.811 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611749+0000", 2024-04-26T04:32:56.812 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00061894499999999998 2024-04-26T04:32:56.812 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.812 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.812 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.812 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611779+0000", 2024-04-26T04:32:56.812 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9969999999999999e-05 2024-04-26T04:32:56.812 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.812 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.812 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.812 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611795+0000", 2024-04-26T04:32:56.812 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5443000000000001e-05 2024-04-26T04:32:56.812 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.812 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.813 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.813 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613563+0000", 2024-04-26T04:32:56.813 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001768211 2024-04-26T04:32:56.813 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.813 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.813 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.813 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613590+0000", 2024-04-26T04:32:56.813 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6633000000000001e-05 2024-04-26T04:32:56.813 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.813 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.813 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.813 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812660+0000", 2024-04-26T04:32:56.813 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19907063799999999 2024-04-26T04:32:56.814 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.814 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.814 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.814 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812755+0000", 2024-04-26T04:32:56.814 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4792999999999999e-05 2024-04-26T04:32:56.814 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.814 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.814 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.814 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.814 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.814 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 40/30 ECSubWrite(tid=106, reqid=client.4670.0:322, at_version=40'19, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:32:56.814 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.613168+0000", 2024-04-26T04:32:56.815 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 22.158551540000001, 2024-04-26T04:32:56.815 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.199657323, 2024-04-26T04:32:56.815 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.815 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.815 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.815 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.815 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.815 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.815 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613168+0000", 2024-04-26T04:32:56.815 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.815 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.815 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.815 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613168+0000", 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613174+0000", 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9569999999999999e-06 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613348+0000", 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000173535 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.816 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613377+0000", 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9522999999999998e-05 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613394+0000", 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.628e-05 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.615501+0000", 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021068710000000002 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.817 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.615545+0000", 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4406999999999997e-05 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812776+0000", 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19723128400000001 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812826+0000", 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9465999999999997e-05 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.818 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 40/30 ECSubWrite(tid=107, reqid=client.4670.0:323, at_version=40'20, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.613975+0000", 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 22.15774532, 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.199514147, 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613975+0000", 2024-04-26T04:32:56.819 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613975+0000", 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613976+0000", 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8029999999999999e-06 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.614012+0000", 2024-04-26T04:32:56.820 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5283000000000001e-05 2024-04-26T04:32:56.821 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.821 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.821 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.821 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.614038+0000", 2024-04-26T04:32:56.821 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5890000000000001e-05 2024-04-26T04:32:56.821 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.821 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.821 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.821 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.614070+0000", 2024-04-26T04:32:56.821 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1992999999999997e-05 2024-04-26T04:32:56.821 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.821 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.822 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.822 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812143+0000", 2024-04-26T04:32:56.822 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.1980729 2024-04-26T04:32:56.822 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.822 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.822 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.822 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812157+0000", 2024-04-26T04:32:56.822 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.472e-05 2024-04-26T04:32:56.822 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.822 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.822 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.822 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.813409+0000", 2024-04-26T04:32:56.822 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012518290000000001 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.813489+0000", 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9728999999999995e-05 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.es5 41/30 ECSubWrite(tid=197, reqid=client.4670.0:349, at_version=41'14, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:35.628948+0000", 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.142772045000001, 2024-04-26T04:32:56.823 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104618195, 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.628948+0000", 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.628948+0000", 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.824 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.628949+0000", 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2399999999999997e-07 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.629254+0000", 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030504300000000001 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.629283+0000", 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.885e-05 2024-04-26T04:32:56.825 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.729430+0000", 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100147071 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.729622+0000", 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000192015 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.729640+0000", 2024-04-26T04:32:56.826 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8202999999999998e-05 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.733492+0000", 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0038524319999999998 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.733566+0000", 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3757000000000006e-05 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.827 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 47/30 ECSubWrite(tid=144, reqid=osd.4.0:144, at_version=47'23, trim_to=0'0, roll_forward_to=46'21))", 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:41.204687+0000", 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 15.567033263000001, 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10505213300000001, 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.204687+0000", 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.828 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.204687+0000", 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.204687+0000", 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9999999999999997e-07 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.204725+0000", 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7595999999999997e-05 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.829 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.204752+0000", 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.669e-05 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.204768+0000", 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6563e-05 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.307554+0000", 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10278620400000001 2024-04-26T04:32:56.830 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.307568+0000", 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3349999999999999e-05 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.309698+0000", 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021302309999999998 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.309739+0000", 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0899000000000003e-05 2024-04-26T04:32:56.831 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.as5 47/30 ECSubWrite(tid=37, reqid=osd.3.0:37, at_version=47'29, trim_to=0'0, roll_forward_to=46'27))", 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:41.268922+0000", 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 15.502798139999999, 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.140330859, 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.832 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.833 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.268922+0000", 2024-04-26T04:32:56.833 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.833 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.833 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.833 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.833 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.268922+0000", 2024-04-26T04:32:56.833 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.833 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.833 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.833 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.833 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.268922+0000", 2024-04-26T04:32:56.833 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5299999999999999e-07 2024-04-26T04:32:56.833 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.834 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.834 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.834 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.268962+0000", 2024-04-26T04:32:56.834 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9589000000000003e-05 2024-04-26T04:32:56.834 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.834 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.834 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.834 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.268983+0000", 2024-04-26T04:32:56.834 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0964000000000001e-05 2024-04-26T04:32:56.834 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.834 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.834 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.834 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.269001+0000", 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8023e-05 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.407874+0000", 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.138872829 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.407902+0000", 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8663e-05 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.835 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.836 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.409171+0000", 2024-04-26T04:32:56.836 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001268885 2024-04-26T04:32:56.836 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.836 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.836 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.836 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.409253+0000", 2024-04-26T04:32:56.836 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1452999999999994e-05 2024-04-26T04:32:56.836 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.836 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.836 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.836 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.836 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.836 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.5s6 56/30 ECSubWrite(tid=63, reqid=client.4670.0:480, at_version=0'0, trim_to=0'0, roll_forward_to=56'44))", 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:50.926782+0000", 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.8449380560000002, 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20180579100000001, 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926782+0000", 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.837 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926782+0000", 2024-04-26T04:32:56.838 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.838 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.838 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.838 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.838 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926784+0000", 2024-04-26T04:32:56.838 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8530000000000001e-06 2024-04-26T04:32:56.838 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.838 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.838 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.838 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926806+0000", 2024-04-26T04:32:56.838 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2104e-05 2024-04-26T04:32:56.838 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.838 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926835+0000", 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9635999999999999e-05 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.026977+0000", 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014184800000001 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.127264+0000", 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100287105 2024-04-26T04:32:56.839 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.127291+0000", 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7046000000000002e-05 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.128508+0000", 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012162760000000001 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.128588+0000", 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9923e-05 2024-04-26T04:32:56.840 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.5s6 57/30 ECSubWrite(tid=65, reqid=osd.15.0:65, at_version=57'46, trim_to=0'0, roll_forward_to=56'44))", 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.522358+0000", 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.2493617659999998, 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10484147000000001, 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.841 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.522358+0000", 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.522358+0000", 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.522359+0000", 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7999999999999996e-07 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.842 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.522413+0000", 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3924000000000001e-05 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.522433+0000", 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0516000000000001e-05 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.522450+0000", 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.721e-05 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.843 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621706+0000", 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.099255766999999995 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621733+0000", 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7107e-05 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.627104+0000", 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.005371163 2024-04-26T04:32:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.627200+0000", 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5303000000000006e-05 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.5s6 58/30 ECSubWrite(tid=67, reqid=osd.15.0:67, at_version=58'48, trim_to=0'0, roll_forward_to=57'46))", 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:52.189352+0000", 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.5823676359999999, 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10545953800000001, 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.845 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.189352+0000", 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.189352+0000", 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.846 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.189353+0000", 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0299999999999998e-07 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.189405+0000", 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1956000000000002e-05 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.189431+0000", 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6080000000000001e-05 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.847 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.189450+0000", 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8357000000000001e-05 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.289668+0000", 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10021857300000001 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.289701+0000", 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2555999999999998e-05 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.848 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.294715+0000", 2024-04-26T04:32:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0050141530000000004 2024-04-26T04:32:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.294812+0000", 2024-04-26T04:32:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.7059999999999996e-05 2024-04-26T04:32:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.es5 58/30 ECSubWrite(tid=211, reqid=client.4670.0:497, at_version=0'0, trim_to=0'0, roll_forward_to=58'25))", 2024-04-26T04:32:56.849 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:52.385727+0000", 2024-04-26T04:32:56.850 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.3859930829999998, 2024-04-26T04:32:56.850 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10464506, 2024-04-26T04:32:56.850 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.850 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.850 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.850 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.850 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.850 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.850 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.385727+0000", 2024-04-26T04:32:56.850 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.850 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.850 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.850 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.851 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.385727+0000", 2024-04-26T04:32:56.851 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.851 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.851 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.851 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.385729+0000", 2024-04-26T04:32:56.851 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7129999999999999e-06 2024-04-26T04:32:56.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.851 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.851 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.851 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.385750+0000", 2024-04-26T04:32:56.851 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1236000000000001e-05 2024-04-26T04:32:56.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.385791+0000", 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1449999999999998e-05 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.385810+0000", 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8393000000000002e-05 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.489152+0000", 2024-04-26T04:32:56.852 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10334228199999999 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.489168+0000", 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6079999999999999e-05 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.490278+0000", 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0011096299999999999 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.853 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.490372+0000", 2024-04-26T04:32:56.854 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4276000000000003e-05 2024-04-26T04:32:56.854 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.854 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.854 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.854 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.854 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.854 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 58/30 ECSubWrite(tid=199, reqid=client.4670.0:503, at_version=58'43, trim_to=0'0, roll_forward_to=58'42))", 2024-04-26T04:32:56.854 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:52.602262+0000", 2024-04-26T04:32:56.854 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.169457575, 2024-04-26T04:32:56.854 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103642394, 2024-04-26T04:32:56.854 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.854 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.855 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.855 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.855 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.855 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.855 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.602262+0000", 2024-04-26T04:32:56.855 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.855 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.855 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.855 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.602262+0000", 2024-04-26T04:32:56.855 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.855 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.602265+0000", 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2139999999999999e-06 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.603028+0000", 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000763709 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.603076+0000", 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7333999999999998e-05 2024-04-26T04:32:56.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.857 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.857 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.857 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.703176+0000", 2024-04-26T04:32:56.857 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10009988 2024-04-26T04:32:56.857 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.857 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.857 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.857 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.703343+0000", 2024-04-26T04:32:56.857 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000167589 2024-04-26T04:32:56.857 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.857 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.857 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.857 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.703373+0000", 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9958999999999999e-05 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.705820+0000", 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002446443 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.705905+0000", 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5266000000000007e-05 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.858 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 60/30 ECSubWrite(tid=222, reqid=client.4670.0:597, at_version=60'46, trim_to=0'0, roll_forward_to=58'43))", 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:55.982118+0000", 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.78960168200000003, 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30298714199999999, 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982118+0000", 2024-04-26T04:32:56.859 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982118+0000", 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982119+0000", 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1969999999999999e-06 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982416+0000", 2024-04-26T04:32:56.860 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000296087 2024-04-26T04:32:56.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.861 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.861 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.861 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982433+0000", 2024-04-26T04:32:56.861 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6986999999999998e-05 2024-04-26T04:32:56.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.861 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.861 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.861 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982444+0000", 2024-04-26T04:32:56.861 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1596e-05 2024-04-26T04:32:56.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.861 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.862 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.862 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.182651+0000", 2024-04-26T04:32:56.862 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200207255 2024-04-26T04:32:56.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.862 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.862 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.862 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.182661+0000", 2024-04-26T04:32:56.862 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0017000000000001e-05 2024-04-26T04:32:56.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.862 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.862 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.862 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.285076+0000", 2024-04-26T04:32:56.862 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102415038 2024-04-26T04:32:56.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.863 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.863 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.863 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.285105+0000", 2024-04-26T04:32:56.863 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8965000000000001e-05 2024-04-26T04:32:56.863 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.863 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.863 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.863 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.863 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 60/30 ECSubWrite(tid=223, reqid=client.4670.0:598, at_version=60'47, trim_to=0'0, roll_forward_to=58'43))", 2024-04-26T04:32:56.863 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:56.083100+0000", 2024-04-26T04:32:56.863 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.68862014199999999, 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202036665, 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083100+0000", 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083100+0000", 2024-04-26T04:32:56.864 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:56.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.865 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.865 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:56.865 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083102+0000", 2024-04-26T04:32:56.865 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2170000000000001e-06 2024-04-26T04:32:56.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.865 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.865 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:56.865 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083219+0000", 2024-04-26T04:32:56.865 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000117049 2024-04-26T04:32:56.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.865 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.865 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:56.866 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083244+0000", 2024-04-26T04:32:56.866 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4471e-05 2024-04-26T04:32:56.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.866 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.866 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:56.866 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.183323+0000", 2024-04-26T04:32:56.866 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100079945 2024-04-26T04:32:56.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.866 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.866 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:56.866 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.183491+0000", 2024-04-26T04:32:56.866 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016799800000000001 2024-04-26T04:32:56.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.867 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.867 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:56.867 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.183505+0000", 2024-04-26T04:32:56.867 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3740999999999999e-05 2024-04-26T04:32:56.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.867 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.867 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:56.867 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.285116+0000", 2024-04-26T04:32:56.867 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10161113500000001 2024-04-26T04:32:56.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:56.867 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:56.867 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:56.867 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.285136+0000", 2024-04-26T04:32:56.868 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0109e-05 2024-04-26T04:32:56.868 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.868 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.868 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.868 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:56.868 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:56.868 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:56.869 INFO:tasks.rados.rados.0.smithi178.stdout:243: finishing write tid 5 to smithi17829308-40 2024-04-26T04:32:56.869 INFO:tasks.rados.rados.0.smithi178.stdout:243: oid 40 updating version 0 to 71 2024-04-26T04:32:56.869 INFO:tasks.rados.rados.0.smithi178.stdout:243: oid 40 version 71 is already newer than 70 2024-04-26T04:32:56.870 INFO:tasks.rados.rados.0.smithi178.stdout:243: oid 40 version 71 is already newer than 69 2024-04-26T04:32:56.870 INFO:tasks.rados.rados.0.smithi178.stdout:243: oid 40 version 71 is already newer than 68 2024-04-26T04:32:56.870 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 40 v 71 (ObjNum 105 snap 13 seq_num 105) dirty exists 2024-04-26T04:32:56.870 INFO:tasks.rados.rados.0.smithi178.stdout:243: left oid 40 (ObjNum 105 snap 13 seq_num 105) 2024-04-26T04:32:56.870 INFO:tasks.rados.rados.0.smithi178.stdout:246: finishing write tid 1 to smithi17829308-35 2024-04-26T04:32:56.870 INFO:tasks.rados.rados.0.smithi178.stdout:237: finishing write tid 3 to smithi17829308-30 2024-04-26T04:32:56.870 INFO:tasks.rados.rados.0.smithi178.stdout:237: finishing write tid 4 to smithi17829308-30 2024-04-26T04:32:56.870 INFO:tasks.rados.rados.0.smithi178.stdout:232: finishing copy_from to smithi17829308-32 2024-04-26T04:32:56.870 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 32 v 2 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-04-26T04:32:56.870 INFO:tasks.rados.rados.0.smithi178.stdout:234: finishing write tid 6 to smithi17829308-15 2024-04-26T04:32:56.870 INFO:tasks.rados.rados.0.smithi178.stdout:234: oid 15 updating version 0 to 29 2024-04-26T04:32:56.870 INFO:tasks.rados.rados.0.smithi178.stdout:234: oid 15 updating version 29 to 32 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:234: oid 15 version 32 is already newer than 31 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:234: oid 15 version 32 is already newer than 28 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:234: oid 15 version 32 is already newer than 30 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 15 v 32 (ObjNum 101 snap 13 seq_num 101) dirty exists 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:234: left oid 15 (ObjNum 101 snap 13 seq_num 101) 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:244: expect (ObjNum 13 snap 0 seq_num 13) 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:247: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:235: finishing write tid 5 to smithi17829308-49 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:235: oid 49 updating version 0 to 31 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:235: oid 49 updating version 31 to 32 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:235: oid 49 updating version 32 to 33 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:235: oid 49 updating version 33 to 34 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 49 v 34 (ObjNum 102 snap 13 seq_num 102) dirty exists 2024-04-26T04:32:56.871 INFO:tasks.rados.rados.0.smithi178.stdout:235: left oid 49 (ObjNum 102 snap 13 seq_num 102) 2024-04-26T04:32:56.872 INFO:tasks.rados.rados.0.smithi178.stdout:233: expect (ObjNum 21 snap 0 seq_num 21) 2024-04-26T04:32:56.872 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T04:32:56.874 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:56.873+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:56.896 INFO:tasks.rados.rados.0.smithi178.stdout:238: expect (ObjNum 36 snap 0 seq_num 36) 2024-04-26T04:32:56.918 INFO:tasks.rados.rados.0.smithi178.stdout:242: finishing write tid 5 to smithi17829308-16 2024-04-26T04:32:56.919 INFO:tasks.rados.rados.0.smithi178.stdout:242: oid 16 updating version 0 to 56 2024-04-26T04:32:56.919 INFO:tasks.rados.rados.0.smithi178.stdout:242: oid 16 updating version 56 to 58 2024-04-26T04:32:56.919 INFO:tasks.rados.rados.0.smithi178.stdout:242: oid 16 updating version 58 to 59 2024-04-26T04:32:56.919 INFO:tasks.rados.rados.0.smithi178.stdout:242: oid 16 version 59 is already newer than 57 2024-04-26T04:32:56.919 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 16 v 59 (ObjNum 104 snap 13 seq_num 104) dirty exists 2024-04-26T04:32:56.919 INFO:tasks.rados.rados.0.smithi178.stdout:242: left oid 16 (ObjNum 104 snap 13 seq_num 104) 2024-04-26T04:32:56.940 INFO:tasks.rados.rados.0.smithi178.stdout:246: finishing write tid 2 to smithi17829308-35 2024-04-26T04:32:56.940 INFO:tasks.rados.rados.0.smithi178.stdout:246: finishing write tid 3 to smithi17829308-35 2024-04-26T04:32:56.941 INFO:tasks.rados.rados.0.smithi178.stdout:246: finishing write tid 4 to smithi17829308-35 2024-04-26T04:32:56.941 INFO:tasks.rados.rados.0.smithi178.stdout:246: finishing write tid 5 to smithi17829308-35 2024-04-26T04:32:56.975 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:56.973+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:56.996 INFO:tasks.rados.rados.0.smithi178.stdout:237: finishing write tid 5 to smithi17829308-30 2024-04-26T04:32:56.996 INFO:tasks.rados.rados.0.smithi178.stdout:237: oid 30 updating version 0 to 48 2024-04-26T04:32:56.996 INFO:tasks.rados.rados.0.smithi178.stdout:237: oid 30 version 48 is already newer than 46 2024-04-26T04:32:56.996 INFO:tasks.rados.rados.0.smithi178.stdout:237: oid 30 updating version 48 to 49 2024-04-26T04:32:56.996 INFO:tasks.rados.rados.0.smithi178.stdout:237: oid 30 version 49 is already newer than 47 2024-04-26T04:32:56.996 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 30 v 49 (ObjNum 103 snap 13 seq_num 103) dirty exists 2024-04-26T04:32:56.996 INFO:tasks.rados.rados.0.smithi178.stdout:237: left oid 30 (ObjNum 103 snap 13 seq_num 103) 2024-04-26T04:32:57.000 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:57.000 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:57.000 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:32:57.001 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:57.014 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T04:32:57.076 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:57.074+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:57.132 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:57.132 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:57.132 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:32:57.132 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:32:57.132 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:57.146 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T04:32:57.146 INFO:tasks.rados.rados.0.smithi178.stdout:246: finishing write tid 6 to smithi17829308-35 2024-04-26T04:32:57.146 INFO:tasks.rados.rados.0.smithi178.stdout:246: oid 35 updating version 0 to 73 2024-04-26T04:32:57.147 INFO:tasks.rados.rados.0.smithi178.stdout:246: oid 35 updating version 73 to 77 2024-04-26T04:32:57.147 INFO:tasks.rados.rados.0.smithi178.stdout:246: oid 35 version 77 is already newer than 76 2024-04-26T04:32:57.147 INFO:tasks.rados.rados.0.smithi178.stdout:246: oid 35 updating version 77 to 79 2024-04-26T04:32:57.147 INFO:tasks.rados.rados.0.smithi178.stdout:246: oid 35 version 79 is already newer than 78 2024-04-26T04:32:57.147 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 35 v 79 (ObjNum 106 snap 13 seq_num 106) dirty exists 2024-04-26T04:32:57.147 INFO:tasks.rados.rados.0.smithi178.stdout:246: left oid 35 (ObjNum 106 snap 13 seq_num 106) 2024-04-26T04:32:57.174 INFO:tasks.rados.rados.0.smithi178.stdout:232: done (15 left) 2024-04-26T04:32:57.174 INFO:tasks.rados.rados.0.smithi178.stdout:233: done (14 left) 2024-04-26T04:32:57.174 INFO:tasks.rados.rados.0.smithi178.stdout:234: done (13 left) 2024-04-26T04:32:57.174 INFO:tasks.rados.rados.0.smithi178.stdout:235: done (12 left) 2024-04-26T04:32:57.174 INFO:tasks.rados.rados.0.smithi178.stdout:237: done (11 left) 2024-04-26T04:32:57.174 INFO:tasks.rados.rados.0.smithi178.stdout:238: done (10 left) 2024-04-26T04:32:57.174 INFO:tasks.rados.rados.0.smithi178.stdout:239: done (9 left) 2024-04-26T04:32:57.174 INFO:tasks.rados.rados.0.smithi178.stdout:242: done (8 left) 2024-04-26T04:32:57.174 INFO:tasks.rados.rados.0.smithi178.stdout:243: done (7 left) 2024-04-26T04:32:57.174 INFO:tasks.rados.rados.0.smithi178.stdout:244: done (6 left) 2024-04-26T04:32:57.175 INFO:tasks.rados.rados.0.smithi178.stdout:246: done (5 left) 2024-04-26T04:32:57.175 INFO:tasks.rados.rados.0.smithi178.stdout:247: done (4 left) 2024-04-26T04:32:57.175 INFO:tasks.rados.rados.0.smithi178.stdout:248: done (3 left) 2024-04-26T04:32:57.175 INFO:tasks.rados.rados.0.smithi178.stdout:249: done (2 left) 2024-04-26T04:32:57.175 INFO:tasks.rados.rados.0.smithi178.stdout:250: done (1 left) 2024-04-26T04:32:57.175 INFO:tasks.rados.rados.0.smithi178.stdout:251: done (0 left) 2024-04-26T04:32:57.175 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 33 current snap is 14 2024-04-26T04:32:57.175 INFO:tasks.rados.rados.0.smithi178.stdout:252: seq_num 107 ranges {540672=638976,1179648=458752,1638400=704512} 2024-04-26T04:32:57.184 INFO:tasks.rados.rados.0.smithi178.stdout:252: writing smithi17829308-33 from 540672 to 1179648 tid 1 2024-04-26T04:32:57.187 INFO:tasks.rados.rados.0.smithi178.stdout:252: writing smithi17829308-33 from 1179648 to 1638400 tid 2 2024-04-26T04:32:57.194 INFO:tasks.rados.rados.0.smithi178.stdout:252: writing smithi17829308-33 from 1638400 to 2342912 tid 3 2024-04-26T04:32:57.194 INFO:tasks.rados.rados.0.smithi178.stdout:253: copy_from oid 18 from oid 13 current snap is 14 2024-04-26T04:32:57.194 INFO:tasks.rados.rados.0.smithi178.stdout:254: snap_create 2024-04-26T04:32:57.194 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:32:57.192+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:32:57.274 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:57.274 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:32:57.274 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:32:57.274 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:32:57.274 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 35/30 ECSubWrite(tid=19, reqid=client.4670.0:189, at_version=0'0, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:28.969326+0000", 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 28.304627011000001, 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101585699, 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.969326+0000", 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.275 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.969326+0000", 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.969328+0000", 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8470000000000001e-06 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.969348+0000", 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0699999999999998e-05 2024-04-26T04:32:57.276 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:28.969379+0000", 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0772999999999998e-05 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.069523+0000", 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014342399999999 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.069732+0000", 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00020916199999999999 2024-04-26T04:32:57.277 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.069789+0000", 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7216000000000002e-05 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.070835+0000", 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001046324 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.070911+0000", 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6253000000000003e-05 2024-04-26T04:32:57.278 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 35/30 ECSubWrite(tid=61, reqid=client.4670.0:211, at_version=0'0, trim_to=0'0, roll_forward_to=35'17))", 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:29.330344+0000", 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 27.943609065, 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105013624, 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.279 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330344+0000", 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330344+0000", 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330346+0000", 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1399999999999998e-06 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.280 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330371+0000", 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5320000000000002e-05 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330403+0000", 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2305999999999999e-05 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330423+0000", 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9179000000000001e-05 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.281 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.433931+0000", 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103508818 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.433962+0000", 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0576e-05 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.435260+0000", 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001297819 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.282 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.435357+0000", 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.7466000000000005e-05 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=63, reqid=client.4670.0:236, at_version=0'0, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:29.738155+0000", 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 27.535797432999999, 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10169805799999999, 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.283 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.738155+0000", 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.738155+0000", 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.738157+0000", 2024-04-26T04:32:57.284 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6729999999999999e-06 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.738178+0000", 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1053e-05 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.738208+0000", 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0017000000000001e-05 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.738228+0000", 2024-04-26T04:32:57.285 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9516000000000001e-05 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.838511+0000", 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10028327400000001 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.838535+0000", 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4326000000000001e-05 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.286 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.839782+0000", 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001247272 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.839853+0000", 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.0926999999999994e-05 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=73, reqid=client.4670.0:259, at_version=36'27, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.257763+0000", 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 27.01618938, 2024-04-26T04:32:57.287 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103026436, 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.257763+0000", 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.257763+0000", 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.257766+0000", 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.216e-06 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258080+0000", 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00031390699999999999 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258110+0000", 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0497e-05 2024-04-26T04:32:57.289 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258127+0000", 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6663e-05 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.358401+0000", 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10027383099999999 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.358415+0000", 2024-04-26T04:32:57.290 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4207e-05 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.360708+0000", 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0022931499999999999 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.360790+0000", 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1965000000000006e-05 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.291 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=74, reqid=client.4670.0:260, at_version=36'28, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.258165+0000", 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 27.015787954, 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10329530100000001, 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258165+0000", 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.292 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.293 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258165+0000", 2024-04-26T04:32:57.293 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.293 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.293 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.293 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.293 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258166+0000", 2024-04-26T04:32:57.293 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8299999999999995e-07 2024-04-26T04:32:57.293 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.293 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.293 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.293 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258269+0000", 2024-04-26T04:32:57.293 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000103069 2024-04-26T04:32:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258288+0000", 2024-04-26T04:32:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8652999999999999e-05 2024-04-26T04:32:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258299+0000", 2024-04-26T04:32:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1243999999999999e-05 2024-04-26T04:32:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.294 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.295 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.359776+0000", 2024-04-26T04:32:57.295 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10147722000000001 2024-04-26T04:32:57.295 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.295 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.295 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.295 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.359798+0000", 2024-04-26T04:32:57.295 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2476999999999998e-05 2024-04-26T04:32:57.295 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.295 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.295 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.295 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.361395+0000", 2024-04-26T04:32:57.295 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0015963660000000001 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.361460+0000", 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5289000000000005e-05 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.2s1 36/30 ECSubWrite(tid=194, reqid=client.4670.0:273, at_version=36'9, trim_to=0'0, roll_forward_to=34'7))", 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.432974+0000", 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.840978655000001, 2024-04-26T04:32:57.296 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10231934299999999, 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.432974+0000", 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.432974+0000", 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.297 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.432975+0000", 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.8400000000000003e-07 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.433021+0000", 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6053000000000001e-05 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.433041+0000", 2024-04-26T04:32:57.298 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.982e-05 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.433058+0000", 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7226000000000001e-05 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.533310+0000", 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100251811 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.299 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.533324+0000", 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4545999999999999e-05 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.535227+0000", 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0019027600000000001 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.535293+0000", 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6242999999999996e-05 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.300 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 40/30 ECSubWrite(tid=105, reqid=client.4670.0:331, at_version=40'34, trim_to=0'0, roll_forward_to=38'33))", 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.612294+0000", 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 22.661658363000001, 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101661354, 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.612294+0000", 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.301 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.302 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.302 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.302 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.612294+0000", 2024-04-26T04:32:57.302 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.302 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.302 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.302 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.302 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.612295+0000", 2024-04-26T04:32:57.302 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.03e-07 2024-04-26T04:32:57.302 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.302 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.302 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.302 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.612323+0000", 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7767e-05 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.612348+0000", 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5256e-05 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.712506+0000", 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100157774 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.303 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.712718+0000", 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00021275800000000001 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.712740+0000", 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1137000000000001e-05 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.713874+0000", 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0011344230000000001 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.304 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.713956+0000", 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1835999999999994e-05 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 40/30 ECSubWrite(tid=32, reqid=client.4670.0:332, at_version=40'19, trim_to=0'0, roll_forward_to=40'17))", 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.920408+0000", 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 22.353545068999999, 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10273331099999999, 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.305 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920408+0000", 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920408+0000", 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920410+0000", 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.813e-06 2024-04-26T04:32:57.306 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920465+0000", 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5549e-05 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.920503+0000", 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7580000000000003e-05 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.020672+0000", 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10016948000000001 2024-04-26T04:32:57.307 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.020836+0000", 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000164036 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.020861+0000", 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4349999999999999e-05 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023056+0000", 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021951800000000001 2024-04-26T04:32:57.308 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023141+0000", 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5322999999999996e-05 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 41/30 ECSubWrite(tid=44, reqid=client.4670.0:346, at_version=41'26, trim_to=0'0, roll_forward_to=38'24))", 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.485664896999999, 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103457721, 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.309 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.310 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4600000000000005e-07 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788918+0000", 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00062948799999999999 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788943+0000", 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4706e-05 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.311 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.889107+0000", 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100164644 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.889285+0000", 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017809600000000001 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.889311+0000", 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5233e-05 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.312 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.891664+0000", 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0023538460000000001 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.891746+0000", 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1162000000000007e-05 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpRead(3.ds3 45/30 ECSubRead(tid=38, to_read={3:b7d7af40:::smithi17829308-48:head=0,200704,0}, subchunks={3:b7d7af40:::smithi17829308-48:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:39.701595+0000", 2024-04-26T04:32:57.313 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 17.572357501999999, 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101901167, 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.701595+0000", 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.701595+0000", 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.314 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.701598+0000", 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6129999999999998e-06 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.701622+0000", 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4562999999999998e-05 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.701662+0000", 2024-04-26T04:32:57.315 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9218999999999998e-05 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.701687+0000", 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5527000000000001e-05 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.803375+0000", 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10168782699999999 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.803496+0000", 2024-04-26T04:32:57.316 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000121418 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpRead(3.ds3 45/30 ECSubRead(tid=39, to_read={3:b7d7af40:::smithi17829308-48:head=0,200704,0}, subchunks={3:b7d7af40:::smithi17829308-48:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:39.702108+0000", 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 17.571844551000002, 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101803459, 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.317 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.702108+0000", 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.702108+0000", 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.702109+0000", 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1999999999999999e-06 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.318 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.702123+0000", 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.359e-05 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.702147+0000", 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3609999999999999e-05 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.702162+0000", 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5440000000000001e-05 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.319 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.803620+0000", 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10145747099999999 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.803912+0000", 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000292148 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/30 ECSubWrite(tid=186, reqid=client.4670.0:483, at_version=57'57, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T04:32:57.320 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.517139+0000", 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.7568142440000001, 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.22991919599999999, 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.517139+0000", 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.517139+0000", 2024-04-26T04:32:57.321 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.517139+0000", 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6700000000000003e-07 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.518064+0000", 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00092505799999999998 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.322 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.518095+0000", 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0929999999999997e-05 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.518120+0000", 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4888999999999998e-05 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.644583+0000", 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.12646262999999999 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.323 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.644598+0000", 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5427e-05 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747001+0000", 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102403228 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747058+0000", 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6366999999999998e-05 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.324 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/30 ECSubWrite(tid=187, reqid=client.4670.0:484, at_version=57'58, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.519747+0000", 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.7542058469999997, 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.22735620400000001, 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.519747+0000", 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.325 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.519747+0000", 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.519764+0000", 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7249999999999999e-05 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.520091+0000", 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000326457 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.326 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.520114+0000", 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3462999999999999e-05 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.520135+0000", 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0449999999999999e-05 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.646545+0000", 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.12641021699999999 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.327 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.646559+0000", 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4137e-05 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747071+0000", 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100512381 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747103+0000", 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1848999999999999e-05 2024-04-26T04:32:57.328 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/30 ECSubWrite(tid=188, reqid=client.4670.0:485, at_version=57'59, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.521190+0000", 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.7527627460000001, 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.225943687, 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.329 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521190+0000", 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521190+0000", 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521191+0000", 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2500000000000001e-06 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.330 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521409+0000", 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000218192 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521431+0000", 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1776e-05 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621545+0000", 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10011374300000001 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.331 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.647483+0000", 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.025938086999999999 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.647497+0000", 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.376e-05 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747108+0000", 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.099611379 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.332 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747134+0000", 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.55e-05 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/30 ECSubWrite(tid=189, reqid=client.4670.0:486, at_version=57'60, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.621601+0000", 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.6523520349999998, 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.126106461, 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.333 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621601+0000", 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621601+0000", 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621602+0000", 2024-04-26T04:32:57.334 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.576e-06 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621645+0000", 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2620000000000002e-05 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621689+0000", 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4502999999999997e-05 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.335 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621712+0000", 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2486000000000001e-05 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.746683+0000", 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.124971128 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.746698+0000", 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5140000000000001e-05 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.336 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747655+0000", 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00095687099999999998 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747707+0000", 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2136999999999999e-05 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 58/30 ECSubWrite(tid=195, reqid=osd.4.0:195, at_version=58'62, trim_to=0'0, roll_forward_to=57'60))", 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:52.382786+0000", 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.8911665859999998, 2024-04-26T04:32:57.337 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20206534200000001, 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382786+0000", 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382786+0000", 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.338 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382787+0000", 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9000000000000002e-07 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382822+0000", 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5889999999999997e-05 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382851+0000", 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8370000000000001e-05 2024-04-26T04:32:57.339 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.482997+0000", 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100146258 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.583261+0000", 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100264055 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.583277+0000", 2024-04-26T04:32:57.340 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5783000000000001e-05 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.584773+0000", 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014958440000000001 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.584852+0000", 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8751999999999994e-05 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.341 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 58/30 ECSubWrite(tid=71, reqid=osd.11.0:71, at_version=58'42, trim_to=0'0, roll_forward_to=57'40))", 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:52.393220+0000", 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.88073275, 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101843682, 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.393220+0000", 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.342 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.393220+0000", 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.393220+0000", 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6699999999999999e-07 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.393256+0000", 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5525999999999998e-05 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.343 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.393283+0000", 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6703000000000002e-05 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.393306+0000", 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3672999999999998e-05 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.493523+0000", 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100217026 2024-04-26T04:32:57.344 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.493544+0000", 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0239999999999999e-05 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.494968+0000", 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001424102 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.495064+0000", 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6045000000000006e-05 2024-04-26T04:32:57.345 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 58/30 ECSubWrite(tid=72, reqid=osd.11.0:72, at_version=58'44, trim_to=0'0, roll_forward_to=57'40))", 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:52.395907+0000", 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.8780461129999999, 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102102893, 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.346 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.395907+0000", 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.395907+0000", 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.395908+0000", 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.773e-06 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.347 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.395960+0000", 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1986e-05 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.396002+0000", 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1103000000000003e-05 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.496069+0000", 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100067584 2024-04-26T04:32:57.348 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.496193+0000", 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012362300000000001 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.496208+0000", 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.556e-05 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.497935+0000", 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001726755 2024-04-26T04:32:57.349 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.498010+0000", 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4509000000000001e-05 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpRead(3.2s1 61/60 ECSubRead(tid=0, to_read={3:4497087f:::smithi17829308-22:head=0,90112,0}, subchunks={3:4497087f:::smithi17829308-22:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:56.532118+0000", 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.74183524499999998, 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101844597, 2024-04-26T04:32:57.350 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.532118+0000", 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.532118+0000", 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.351 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.532118+0000", 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.532118+0000", 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.532164+0000", 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6779000000000001e-05 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.352 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.532176+0000", 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1188e-05 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.633866+0000", 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101690981 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.633962+0000", 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5649000000000005e-05 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.353 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpReadReply(3.2s1 61/60 ECSubReadReply(tid=2, attrs_read=0))", 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:56.636057+0000", 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 0.63789618299999995, 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200517522, 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.354 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636057+0000", 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636057+0000", 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636058+0000", 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1429999999999999e-06 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.355 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636180+0000", 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000122409 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636199+0000", 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8536999999999999e-05 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.736272+0000", 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100073096 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.356 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.836499+0000", 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10022764100000001 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.836574+0000", 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4696000000000004e-05 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.357 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:57.359 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T04:32:57.360 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:32:57.358+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:32:57.460 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:57.458+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:57.529 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:57.529 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:57.529 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:32:57.529 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:57.544 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T04:32:57.590 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:57.588+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:57.610 INFO:tasks.rados.rados.0.smithi178.stdout:253: finishing copy_from racing read to smithi17829308-18 2024-04-26T04:32:57.662 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:57.662 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:32:57.662 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:32:57.662 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:32:57.662 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:57.663 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:57.658+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:57.676 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T04:32:57.704 INFO:tasks.rados.rados.0.smithi178.stdout:252: finishing write tid 1 to smithi17829308-33 2024-04-26T04:32:57.704 INFO:tasks.rados.rados.0.smithi178.stdout:252: finishing write tid 2 to smithi17829308-33 2024-04-26T04:32:57.705 INFO:tasks.rados.rados.0.smithi178.stdout:252: finishing write tid 3 to smithi17829308-33 2024-04-26T04:32:57.711 INFO:tasks.rados.rados.0.smithi178.stdout:253: finishing copy_from to smithi17829308-18 2024-04-26T04:32:57.711 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 18 v 49 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-04-26T04:32:57.763 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:57.762+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:57.843 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:32:57.843 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:32:57.844 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:32:57.844 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:32:57.844 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.844 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:22 3.0s0 3:0061323f:::smithi17829308-8:head [append 0~131072,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:57.844 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.206381+0000", 2024-04-26T04:32:57.844 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.637542889000002, 2024-04-26T04:32:57.844 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30553245699999998, 2024-04-26T04:32:57.844 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.844 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.844 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:57.844 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:32:57.844 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:32:57.844 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 22 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206381+0000", 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206381+0000", 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.845 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206385+0000", 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3299999999999999e-06 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206772+0000", 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000387673 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206797+0000", 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4156e-05 2024-04-26T04:32:57.846 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.847 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.847 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.847 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206811+0000", 2024-04-26T04:32:57.847 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4087000000000001e-05 2024-04-26T04:32:57.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.847 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.847 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.847 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206881+0000", 2024-04-26T04:32:57.847 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9948000000000002e-05 2024-04-26T04:32:57.847 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.847 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.847 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:57.847 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206974+0000", 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.3047000000000005e-05 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207449+0000", 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00047526900000000001 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208764+0000", 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001315395 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:57.848 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511723+0000", 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.302958377 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511914+0000", 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000191175 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:23 3.0s0 3:0061323f:::smithi17829308-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.206854+0000", 2024-04-26T04:32:57.849 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.637070373, 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.407632722, 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 23 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206854+0000", 2024-04-26T04:32:57.850 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206854+0000", 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206855+0000", 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.133e-06 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206875+0000", 2024-04-26T04:32:57.851 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9945999999999998e-05 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206889+0000", 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3997e-05 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206899+0000", 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8490000000000001e-06 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207769+0000", 2024-04-26T04:32:57.852 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00087023199999999997 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207895+0000", 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012591300000000001 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208201+0000", 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030561 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.853 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.209436+0000", 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012354969999999999 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.614327+0000", 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.40489055000000002 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.614487+0000", 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015999499999999999 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.854 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:24 3.0s0 3:0061323f:::smithi17829308-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.206918+0000", 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.637006581000001, 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.612165556, 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 24 2024-04-26T04:32:57.855 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206918+0000", 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206918+0000", 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.856 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206919+0000", 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5000000000000001e-07 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206923+0000", 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0330000000000002e-06 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206934+0000", 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0866999999999999e-05 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.857 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206946+0000", 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2947e-05 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208374+0000", 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014272930000000001 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208613+0000", 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00023916200000000001 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.858 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.615561+0000", 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.40694828199999999 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.615825+0000", 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00026369700000000002 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.819083+0000", 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20325832499999999 2024-04-26T04:32:57.859 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=0, reqid=client.4670.0:1, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.541077567000002, 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.208159126, 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.860 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8299999999999998e-07 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.861 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302898+0000", 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1421000000000001e-05 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302912+0000", 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4075e-05 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302928+0000", 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5333999999999999e-05 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.862 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403201+0000", 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100273162 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403219+0000", 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7526000000000001e-05 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.510967+0000", 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107748329 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.863 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511006+0000", 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8995999999999997e-05 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=1, reqid=client.4670.0:2, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.540974489999996, 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.208096686, 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.864 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.865 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.82e-07 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302987+0000", 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6507999999999997e-05 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302996+0000", 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4900000000000006e-06 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.866 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303002+0000", 2024-04-26T04:32:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3800000000000002e-06 2024-04-26T04:32:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403906+0000", 2024-04-26T04:32:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100904445 2024-04-26T04:32:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403919+0000", 2024-04-26T04:32:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.259e-05 2024-04-26T04:32:57.867 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511012+0000", 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107093034 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511046+0000", 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4756999999999999e-05 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.868 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=2, reqid=client.4670.0:13, at_version=32'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.303011+0000", 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.540913025999998, 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20833385600000001, 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303011+0000", 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.869 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303011+0000", 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303012+0000", 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4700000000000002e-07 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303033+0000", 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1123000000000001e-05 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.870 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303040+0000", 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5340000000000002e-06 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303044+0000", 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1620000000000001e-06 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.404280+0000", 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101235978 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.871 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.404303+0000", 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2446000000000002e-05 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511267+0000", 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.1069645 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511345+0000", 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7766e-05 2024-04-26T04:32:57.872 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 32/30 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.305887+0000", 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.538037467999999, 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20516263700000001, 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.873 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.874 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305887+0000", 2024-04-26T04:32:57.874 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.874 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.874 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.874 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305887+0000", 2024-04-26T04:32:57.874 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.874 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.874 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.874 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305888+0000", 2024-04-26T04:32:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.108e-06 2024-04-26T04:32:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305895+0000", 2024-04-26T04:32:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2099999999999996e-06 2024-04-26T04:32:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305914+0000", 2024-04-26T04:32:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9164000000000002e-05 2024-04-26T04:32:57.875 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305925+0000", 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1156e-05 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.510931+0000", 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20500514 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511049+0000", 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00011885899999999999 2024-04-26T04:32:57.876 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 32/30 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.306725+0000", 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.537199102000002, 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20445080600000001, 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.877 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306725+0000", 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306725+0000", 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306726+0000", 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2500000000000004e-07 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.878 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306731+0000", 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9250000000000003e-06 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306745+0000", 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4367e-05 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306753+0000", 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9699999999999999e-06 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.879 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511144+0000", 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20439093 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511176+0000", 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1689000000000001e-05 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs2 32/30 ECSubWrite(tid=1, reqid=client.4670.0:8, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:57.880 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.344461+0000", 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.499463511999998, 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10304110700000001, 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.344461+0000", 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.344461+0000", 2024-04-26T04:32:57.881 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.344461+0000", 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3599999999999999e-07 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.344496+0000", 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5179999999999999e-05 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.882 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.344521+0000", 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4766e-05 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.444630+0000", 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100108871 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.444733+0000", 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000103014 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.883 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.444745+0000", 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.204e-05 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.447426+0000", 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0026813309999999999 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.447502+0000", 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5469000000000005e-05 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.884 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 33/30 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.397734+0000", 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.446190037999997, 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.113617151, 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397734+0000", 2024-04-26T04:32:57.885 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397734+0000", 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397736+0000", 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.933e-06 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.886 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397748+0000", 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1433999999999999e-05 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397769+0000", 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1115999999999999e-05 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397794+0000", 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5426999999999999e-05 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.887 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.888 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511242+0000", 2024-04-26T04:32:57.888 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.11344815499999999 2024-04-26T04:32:57.888 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.888 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.888 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.888 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511351+0000", 2024-04-26T04:32:57.888 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010908600000000001 2024-04-26T04:32:57.888 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.888 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.888 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.888 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.888 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.888 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 33/30 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.397821+0000", 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.446103723999997, 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.11354435, 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397821+0000", 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397821+0000", 2024-04-26T04:32:57.889 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.890 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.890 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.890 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.890 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397821+0000", 2024-04-26T04:32:57.890 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.44e-07 2024-04-26T04:32:57.890 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.890 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.890 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.890 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397825+0000", 2024-04-26T04:32:57.890 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8039999999999999e-06 2024-04-26T04:32:57.890 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.890 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.890 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397836+0000", 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1292e-05 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397844+0000", 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.43e-06 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511322+0000", 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.11347817 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.891 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.892 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.892 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511365+0000", 2024-04-26T04:32:57.892 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3109999999999999e-05 2024-04-26T04:32:57.892 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.892 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.892 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.892 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.892 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.892 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=3, reqid=client.4670.0:14, at_version=32'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:32:57.892 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.403652+0000", 2024-04-26T04:32:57.892 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.440271836999997, 2024-04-26T04:32:57.892 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.109359838, 2024-04-26T04:32:57.892 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403652+0000", 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403652+0000", 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.893 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403653+0000", 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8099999999999999e-07 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403676+0000", 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3708999999999999e-05 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403695+0000", 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8754e-05 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.894 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.895 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.895 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403710+0000", 2024-04-26T04:32:57.895 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4577e-05 2024-04-26T04:32:57.895 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.895 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.895 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.895 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511469+0000", 2024-04-26T04:32:57.895 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107758928 2024-04-26T04:32:57.895 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.895 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.895 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.895 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511490+0000", 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1036999999999999e-05 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.512951+0000", 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00146148 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.513012+0000", 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.1072000000000001e-05 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.896 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 35/30 ECSubWrite(tid=34, reqid=client.4670.0:202, at_version=35'24, trim_to=0'0, roll_forward_to=35'22))", 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:29.161802+0000", 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 28.682122034999999, 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103248995, 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.161802+0000", 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.897 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.898 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.898 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.898 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.161802+0000", 2024-04-26T04:32:57.898 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.898 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.898 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.898 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.898 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.161804+0000", 2024-04-26T04:32:57.898 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9539999999999998e-06 2024-04-26T04:32:57.898 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.898 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.898 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.162608+0000", 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00080376299999999998 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.162641+0000", 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2570000000000002e-05 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.162665+0000", 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4939999999999998e-05 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.899 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.900 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.262947+0000", 2024-04-26T04:32:57.900 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10028109 2024-04-26T04:32:57.900 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.900 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.900 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.900 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.262963+0000", 2024-04-26T04:32:57.900 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6152999999999999e-05 2024-04-26T04:32:57.900 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.900 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.900 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.900 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.264973+0000", 2024-04-26T04:32:57.900 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020107390000000001 2024-04-26T04:32:57.900 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.265051+0000", 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7786000000000004e-05 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:280 3.0s0 3:0061323f:::smithi17829308-8:head [delete] snapc 2=[2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.865956+0000", 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.977968135000001, 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20424141400000001, 2024-04-26T04:32:57.901 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 280 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865956+0000", 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.902 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865956+0000", 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865959+0000", 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0570000000000001e-06 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865971+0000", 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1576000000000001e-05 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.903 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.904 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.904 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866003+0000", 2024-04-26T04:32:57.904 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2543999999999999e-05 2024-04-26T04:32:57.904 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.904 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.904 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.904 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866020+0000", 2024-04-26T04:32:57.904 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6433e-05 2024-04-26T04:32:57.904 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.904 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.904 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.904 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866157+0000", 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013767200000000001 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866327+0000", 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016995199999999999 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.867476+0000", 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0011486700000000001 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.905 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.869167+0000", 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016912450000000001 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.070080+0000", 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200912916 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.070198+0000", 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000117349 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.906 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.as4 37/30 ECSubWrite(tid=19, reqid=client.4670.0:285, at_version=37'12, trim_to=0'0, roll_forward_to=36'10))", 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:31.258576+0000", 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.585347799000001, 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103210529, 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.258576+0000", 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.258576+0000", 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.258577+0000", 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.53e-07 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.259163+0000", 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00058609799999999996 2024-04-26T04:32:57.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.259193+0000", 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9623000000000001e-05 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.259209+0000", 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6303999999999998e-05 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.359488+0000", 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100279481 2024-04-26T04:32:57.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.359503+0000", 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4514e-05 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.361727+0000", 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0022245870000000001 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.361787+0000", 2024-04-26T04:32:57.910 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9568999999999998e-05 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds2 40/30 ECSubWrite(tid=33, reqid=client.4670.0:332, at_version=0'0, trim_to=0'0, roll_forward_to=40'19))", 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:35.023904+0000", 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 22.820019912999999, 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201717813, 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.911 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023904+0000", 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023904+0000", 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023906+0000", 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.387e-06 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.912 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023925+0000", 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9059000000000002e-05 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023952+0000", 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7455999999999999e-05 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023968+0000", 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.526e-05 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.913 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.224267+0000", 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200299636 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.224282+0000", 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5007e-05 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.225547+0000", 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012650719999999999 2024-04-26T04:32:57.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.225622+0000", 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4936000000000002e-05 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.4s5 45/30 ECSubWrite(tid=34, reqid=client.4670.0:377, at_version=45'23, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:39.634866+0000", 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 18.209058284000001, 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20147746799999999, 2024-04-26T04:32:57.915 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634866+0000", 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634866+0000", 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.916 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634866+0000", 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8599999999999999e-07 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634876+0000", 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8649999999999996e-06 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634884+0000", 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7060000000000004e-06 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.917 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.734989+0000", 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100104926 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.835213+0000", 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100223672 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.835223+0000", 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0570999999999999e-05 2024-04-26T04:32:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.836270+0000", 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0010470220000000001 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.836343+0000", 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3319999999999996e-05 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.919 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds2 52/30 ECSubWrite(tid=52, reqid=osd.7.0:52, at_version=52'34, trim_to=0'0, roll_forward_to=50'32))", 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:46.171817+0000", 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 11.672107197999999, 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.199021011, 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.171817+0000", 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.920 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.171817+0000", 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.171817+0000", 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8000000000000001e-07 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.171854+0000", 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6297000000000002e-05 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.921 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.171880+0000", 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5769999999999999e-05 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.171897+0000", 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7646e-05 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.369183+0000", 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.197285603 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.922 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.369219+0000", 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6699000000000002e-05 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.370728+0000", 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0015088269999999999 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.370838+0000", 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000109789 2024-04-26T04:32:57.923 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.4s5 52/30 ECSubWrite(tid=56, reqid=client.4670.0:437, at_version=52'45, trim_to=0'0, roll_forward_to=50'41))", 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:46.236130+0000", 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 11.607794648, 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103527494, 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.924 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.236130+0000", 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.236130+0000", 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.236132+0000", 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9259999999999999e-06 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.925 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.236203+0000", 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1069999999999995e-05 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.236232+0000", 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9856e-05 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.336330+0000", 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100097889 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.926 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.336468+0000", 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013749500000000001 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.336483+0000", 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4986e-05 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.339565+0000", 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0030825729999999999 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.927 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.339657+0000", 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.1699000000000004e-05 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs2 57/30 ECSubWrite(tid=69, reqid=client.4670.0:488, at_version=57'40, trim_to=0'0, roll_forward_to=52'38))", 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.821323+0000", 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 6.0226012259999999, 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104031451, 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:32:57.928 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.821323+0000", 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.821323+0000", 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:32:57.929 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.821325+0000", 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.193e-06 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.822157+0000", 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00083224299999999998 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.822192+0000", 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4900000000000001e-05 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:32:57.930 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.822209+0000", 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6535999999999999e-05 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.922468+0000", 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100259151 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.922484+0000", 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5556000000000001e-05 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.931 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:32:57.932 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.925279+0000", 2024-04-26T04:32:57.932 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0027954989999999999 2024-04-26T04:32:57.932 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:32:57.932 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:32:57.932 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:32:57.932 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.925355+0000", 2024-04-26T04:32:57.932 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5372999999999998e-05 2024-04-26T04:32:57.932 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.932 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.932 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.932 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:32:57.932 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:32:57.932 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:32:57.934 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T04:32:57.934 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:57.933+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:58.011 INFO:tasks.rados.rados.0.smithi178.stdout:252: finishing write tid 4 to smithi17829308-33 2024-04-26T04:32:58.035 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:58.033+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:58.066 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:58.066 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 0 2024-04-26T04:32:58.066 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:58.081 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T04:32:58.135 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:32:58.132+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:32:58.184 INFO:tasks.rados.rados.0.smithi178.stdout:253: done (2 left) 2024-04-26T04:32:58.185 INFO:tasks.rados.rados.0.smithi178.stdout:254: done (1 left) 2024-04-26T04:32:58.185 INFO:tasks.rados.rados.0.smithi178.stdout:255: snap_create 2024-04-26T04:32:58.197 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-26T04:32:58.206 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:58.206 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:58.206 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:32:58.206 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:32:58.206 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:58.220 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T04:32:58.236 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:32:58.232+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:32:58.337 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:32:58.337+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:32:58.346 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:58.347 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:32:58.347 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:32:58.347 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:32:58.347 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.347 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:263 3.as0 3:51a91c23:::smithi17829308-18:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:58.347 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.817670+0000", 2024-04-26T04:32:58.347 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 28.525527512, 2024-04-26T04:32:58.347 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30863979699999999, 2024-04-26T04:32:58.347 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.347 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.347 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:58.347 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.347 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 263 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817670+0000", 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817670+0000", 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.349 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.349 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.349 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817673+0000", 2024-04-26T04:32:58.349 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9560000000000002e-06 2024-04-26T04:32:58.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.349 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.349 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.349 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817687+0000", 2024-04-26T04:32:58.349 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3923e-05 2024-04-26T04:32:58.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.349 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.350 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.350 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817719+0000", 2024-04-26T04:32:58.350 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2231999999999999e-05 2024-04-26T04:32:58.350 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.350 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.350 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.350 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817738+0000", 2024-04-26T04:32:58.350 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8473000000000001e-05 2024-04-26T04:32:58.350 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.350 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.350 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.351 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817834+0000", 2024-04-26T04:32:58.351 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.5913000000000001e-05 2024-04-26T04:32:58.351 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.351 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.351 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.351 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817971+0000", 2024-04-26T04:32:58.351 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013697899999999999 2024-04-26T04:32:58.351 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.351 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.352 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.352 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.818954+0000", 2024-04-26T04:32:58.352 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00098307899999999998 2024-04-26T04:32:58.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.352 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.352 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.352 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.924736+0000", 2024-04-26T04:32:58.352 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10578224899999999 2024-04-26T04:32:58.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.353 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.353 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:58.353 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.126181+0000", 2024-04-26T04:32:58.353 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20144535999999999 2024-04-26T04:32:58.353 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.353 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.353 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.353 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.126310+0000", 2024-04-26T04:32:58.353 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012863300000000001 2024-04-26T04:32:58.353 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.353 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.353 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.353 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.354 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.354 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:264 3.as0 3:51a91c23:::smithi17829308-18:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:58.354 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.817774+0000", 2024-04-26T04:32:58.354 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 28.525423686, 2024-04-26T04:32:58.354 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30889035999999997, 2024-04-26T04:32:58.354 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.354 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.354 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:58.354 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.354 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.354 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.354 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 264 2024-04-26T04:32:58.354 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817774+0000", 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817774+0000", 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817775+0000", 2024-04-26T04:32:58.355 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8299999999999995e-07 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817781+0000", 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9270000000000002e-06 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817793+0000", 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2113000000000001e-05 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817802+0000", 2024-04-26T04:32:58.356 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8000000000000004e-06 2024-04-26T04:32:58.357 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.357 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.357 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.357 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819893+0000", 2024-04-26T04:32:58.357 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0020906169999999999 2024-04-26T04:32:58.357 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.357 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.357 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.357 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.820146+0000", 2024-04-26T04:32:58.357 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000253415 2024-04-26T04:32:58.357 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.357 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.357 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.358 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.821588+0000", 2024-04-26T04:32:58.358 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001441821 2024-04-26T04:32:58.358 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.358 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.358 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.358 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.924739+0000", 2024-04-26T04:32:58.358 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103150835 2024-04-26T04:32:58.358 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.358 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.358 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:58.358 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.126551+0000", 2024-04-26T04:32:58.358 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20181190500000001 2024-04-26T04:32:58.358 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.126665+0000", 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011394400000000001 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:265 3.as0 3:51a91c23:::smithi17829308-18:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.817865+0000", 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 28.525332651999999, 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50897067299999998, 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.359 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:58.360 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.360 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.360 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.360 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 265 2024-04-26T04:32:58.360 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.360 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.360 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.360 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.360 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817865+0000", 2024-04-26T04:32:58.360 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.360 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.360 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.360 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817865+0000", 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817867+0000", 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0999999999999998e-06 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817876+0000", 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8060000000000001e-06 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.361 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.362 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.362 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817899+0000", 2024-04-26T04:32:58.362 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2716000000000001e-05 2024-04-26T04:32:58.362 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.362 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.362 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.362 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817910+0000", 2024-04-26T04:32:58.362 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1579999999999999e-05 2024-04-26T04:32:58.362 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.362 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.362 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.362 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.822433+0000", 2024-04-26T04:32:58.362 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0045222869999999998 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.822747+0000", 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00031401299999999998 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.823790+0000", 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00104357 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.363 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.924740+0000", 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100949605 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.326718+0000", 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40197800900000002 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.326836+0000", 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011798699999999999 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.364 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:328 3.as0 3:527650bd:::smithi17829308-27:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.408149+0000", 2024-04-26T04:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 23.935049320000001, 2024-04-26T04:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.409867282, 2024-04-26T04:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 328 2024-04-26T04:32:58.365 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.366 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.366 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.366 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.366 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408149+0000", 2024-04-26T04:32:58.366 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.366 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.366 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.366 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.366 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408149+0000", 2024-04-26T04:32:58.366 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.366 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.366 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408152+0000", 2024-04-26T04:32:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.8870000000000002e-06 2024-04-26T04:32:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408172+0000", 2024-04-26T04:32:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.912e-05 2024-04-26T04:32:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408216+0000", 2024-04-26T04:32:58.367 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.4370999999999999e-05 2024-04-26T04:32:58.368 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.368 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.368 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.368 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408238+0000", 2024-04-26T04:32:58.368 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1820000000000001e-05 2024-04-26T04:32:58.368 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.368 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.368 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.368 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408355+0000", 2024-04-26T04:32:58.368 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000116882 2024-04-26T04:32:58.368 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.368 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.368 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408364+0000", 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.4099999999999997e-06 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.412080+0000", 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.003715566 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.412245+0000", 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000165914 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.369 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413482+0000", 2024-04-26T04:32:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0012361690000000001 2024-04-26T04:32:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616957+0000", 2024-04-26T04:32:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20347567 2024-04-26T04:32:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.817777+0000", 2024-04-26T04:32:58.370 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20081958 2024-04-26T04:32:58.371 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.371 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.371 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.371 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.818016+0000", 2024-04-26T04:32:58.371 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000238893 2024-04-26T04:32:58.371 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.371 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.371 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.371 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.371 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.371 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:329 3.as0 3:527650bd:::smithi17829308-27:head [rollback 2] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:58.371 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.408276+0000", 2024-04-26T04:32:58.371 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 23.934921416000002, 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61054785099999997, 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 329 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408276+0000", 2024-04-26T04:32:58.372 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408276+0000", 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408277+0000", 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.7999999999999993e-07 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.373 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408284+0000", 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.0389999999999999e-06 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408311+0000", 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6452999999999999e-05 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408324+0000", 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3259e-05 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.374 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408439+0000", 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000114789 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408444+0000", 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.3700000000000003e-06 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.414140+0000", 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0056952110000000004 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.375 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.414263+0000", 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012392699999999999 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.415016+0000", 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00075301600000000002 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616959+0000", 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20194278800000001 2024-04-26T04:32:58.376 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018750+0000", 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40179092999999999 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018824+0000", 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.4089000000000002e-05 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.377 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.378 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:330 3.as0 3:527650bd:::smithi17829308-27:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:58.378 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.408341+0000", 2024-04-26T04:32:58.378 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 23.934856581999998, 2024-04-26T04:32:58.378 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61070586699999996, 2024-04-26T04:32:58.378 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.378 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.378 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:58.378 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.378 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.378 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.378 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 330 2024-04-26T04:32:58.379 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.379 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.379 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.379 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.379 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408341+0000", 2024-04-26T04:32:58.379 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.379 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.379 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.379 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.379 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408341+0000", 2024-04-26T04:32:58.379 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.379 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.379 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.380 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.380 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408342+0000", 2024-04-26T04:32:58.380 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.6700000000000003e-07 2024-04-26T04:32:58.380 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.380 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.380 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.380 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408346+0000", 2024-04-26T04:32:58.380 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7299999999999999e-06 2024-04-26T04:32:58.380 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.380 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.380 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408357+0000", 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1276e-05 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408371+0000", 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3763e-05 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408472+0000", 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010149600000000001 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.381 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.382 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:58.382 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408477+0000", 2024-04-26T04:32:58.382 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5029999999999999e-06 2024-04-26T04:32:58.382 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.382 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.382 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.382 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.515978+0000", 2024-04-26T04:32:58.382 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.107501546 2024-04-26T04:32:58.382 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.382 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.382 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.382 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.516082+0000", 2024-04-26T04:32:58.383 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010331 2024-04-26T04:32:58.383 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.383 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.383 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.383 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.516573+0000", 2024-04-26T04:32:58.383 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000491316 2024-04-26T04:32:58.383 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.383 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.383 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.383 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.617004+0000", 2024-04-26T04:32:58.383 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100431009 2024-04-26T04:32:58.383 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.384 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.384 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:58.384 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018996+0000", 2024-04-26T04:32:58.384 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40199249199999998 2024-04-26T04:32:58.384 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.384 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.384 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.384 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.019047+0000", 2024-04-26T04:32:58.384 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0658999999999997e-05 2024-04-26T04:32:58.384 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.384 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.384 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.384 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.385 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.385 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=27, last_complete=40'19, committed=1, applied=1))", 2024-04-26T04:32:58.385 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.518317+0000", 2024-04-26T04:32:58.385 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 23.824880567000001, 2024-04-26T04:32:58.385 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39993706099999998, 2024-04-26T04:32:58.385 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.385 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.385 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.385 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.385 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.385 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.385 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518317+0000", 2024-04-26T04:32:58.386 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.386 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.386 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.386 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.386 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518317+0000", 2024-04-26T04:32:58.386 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.386 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.386 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.386 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.386 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518319+0000", 2024-04-26T04:32:58.386 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7099999999999999e-06 2024-04-26T04:32:58.386 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.386 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518329+0000", 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0159999999999999e-05 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518362+0000", 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3321999999999997e-05 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518380+0000", 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7693000000000002e-05 2024-04-26T04:32:58.387 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.918198+0000", 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.399817537 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.918254+0000", 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.6638999999999998e-05 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.388 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.518641+0000", 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 23.824556456, 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39974739300000001, 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518641+0000", 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.389 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.390 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.390 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518641+0000", 2024-04-26T04:32:58.390 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.390 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.390 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.390 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.390 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518643+0000", 2024-04-26T04:32:58.390 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9659999999999999e-06 2024-04-26T04:32:58.390 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.390 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.390 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.390 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518656+0000", 2024-04-26T04:32:58.391 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2945999999999999e-05 2024-04-26T04:32:58.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.391 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.391 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518692+0000", 2024-04-26T04:32:58.391 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.5485999999999999e-05 2024-04-26T04:32:58.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.391 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.391 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.518716+0000", 2024-04-26T04:32:58.391 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3906000000000001e-05 2024-04-26T04:32:58.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.918350+0000", 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39963400700000001 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.918389+0000", 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.9082000000000003e-05 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:32:58.392 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.520051+0000", 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 23.823146457, 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49885426900000002, 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520051+0000", 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.393 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520051+0000", 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520052+0000", 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.5000000000000001e-07 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520062+0000", 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9799999999999993e-06 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.394 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520079+0000", 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6518999999999999e-05 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520087+0000", 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.5329999999999994e-06 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018885+0000", 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49879782299999997 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.396 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.396 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018906+0000", 2024-04-26T04:32:58.396 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0564000000000002e-05 2024-04-26T04:32:58.396 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.396 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.396 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.396 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.396 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:32:58.396 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.522047+0000", 2024-04-26T04:32:58.396 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 23.821150771999999, 2024-04-26T04:32:58.396 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49659926700000001, 2024-04-26T04:32:58.396 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522047+0000", 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522047+0000", 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.397 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.398 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522048+0000", 2024-04-26T04:32:58.398 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1200000000000001e-07 2024-04-26T04:32:58.398 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.398 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.398 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522054+0000", 2024-04-26T04:32:58.398 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.049e-06 2024-04-26T04:32:58.398 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.398 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.398 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522070+0000", 2024-04-26T04:32:58.398 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5492999999999999e-05 2024-04-26T04:32:58.398 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522079+0000", 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1679999999999993e-06 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018623+0000", 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49654389700000001 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018646+0000", 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3748e-05 2024-04-26T04:32:58.399 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.400 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.400 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.400 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=27, last_complete=40'19, committed=1, applied=1))", 2024-04-26T04:32:58.400 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.616517+0000", 2024-04-26T04:32:58.400 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 23.726680927, 2024-04-26T04:32:58.400 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40232084899999998, 2024-04-26T04:32:58.400 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.400 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.400 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.400 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.401 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.401 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616517+0000", 2024-04-26T04:32:58.401 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.401 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.401 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616517+0000", 2024-04-26T04:32:58.401 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.401 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.401 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616518+0000", 2024-04-26T04:32:58.401 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.0999999999999997e-07 2024-04-26T04:32:58.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.402 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.402 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616523+0000", 2024-04-26T04:32:58.402 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0490000000000003e-06 2024-04-26T04:32:58.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.402 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.402 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616539+0000", 2024-04-26T04:32:58.402 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6555999999999999e-05 2024-04-26T04:32:58.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.402 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616549+0000", 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8269999999999993e-06 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018674+0000", 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40212457600000001 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018838+0000", 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000164031 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.403 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.616951+0000", 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 23.726247158, 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.402569169, 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.404 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616951+0000", 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616951+0000", 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616951+0000", 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.8700000000000005e-07 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.405 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.406 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616957+0000", 2024-04-26T04:32:58.406 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.3589999999999999e-06 2024-04-26T04:32:58.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.406 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.406 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616974+0000", 2024-04-26T04:32:58.406 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7278999999999998e-05 2024-04-26T04:32:58.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.406 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.406 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616983+0000", 2024-04-26T04:32:58.406 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9500000000000007e-06 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018945+0000", 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40196159399999998 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.019520+0000", 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0005752 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:333 3.as0 3:527650bd:::smithi17829308-27:head [copy-from ver 2] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.120049+0000", 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 23.223149037999999, 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.66333988200000005, 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 333 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120049+0000", 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120049+0000", 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120050+0000", 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.536e-06 2024-04-26T04:32:58.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120057+0000", 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6869999999999997e-06 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120074+0000", 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6787e-05 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120086+0000", 2024-04-26T04:32:58.410 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2665e-05 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.220306+0000", 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10021996900000001 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.220485+0000", 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017836800000000001 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.411 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.577412+0000", 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.35692740299999998 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.580481+0000", 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.003068735 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.783160+0000", 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20267940200000001 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.412 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.783389+0000", 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022833 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:334 3.as0 3:527650bd:::smithi17829308-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.120106+0000", 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 23.223091555, 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.459292215, 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:58.413 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.414 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.414 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.414 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 334 2024-04-26T04:32:58.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.414 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.414 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.414 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120106+0000", 2024-04-26T04:32:58.414 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.414 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.414 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120106+0000", 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120107+0000", 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.540000000000001e-07 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120111+0000", 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.6160000000000002e-06 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.415 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120124+0000", 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2965e-05 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.220210+0000", 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100086047 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221197+0000", 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00098664200000000003 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.416 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221256+0000", 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9734000000000001e-05 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.578868+0000", 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.35761166799999999 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579093+0000", 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022499099999999999 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.417 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.418 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579399+0000", 2024-04-26T04:32:58.418 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00030559800000000002 2024-04-26T04:32:58.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.418 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.418 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:620 3.as0 3:57b6ca82:::smithi17829308-14:b [read 0~229376,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:32:58.418 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:55.807425+0000", 2024-04-26T04:32:58.418 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 2.5357730599999999, 2024-04-26T04:32:58.418 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50556072699999999, 2024-04-26T04:32:58.418 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 620 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807425+0000", 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.419 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807425+0000", 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807429+0000", 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.0459999999999999e-06 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807448+0000", 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9553000000000001e-05 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.420 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807498+0000", 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.9369000000000001e-05 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807524+0000", 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5843e-05 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907865+0000", 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100341135 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.421 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.422 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.908044+0000", 2024-04-26T04:32:58.422 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017972100000000001 2024-04-26T04:32:58.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.422 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.422 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.312986+0000", 2024-04-26T04:32:58.422 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40494106000000002 2024-04-26T04:32:58.422 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.422 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.422 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.423 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:621 3.as0 3:57b6ca82:::smithi17829308-14:b [read 0~229376,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:32:58.423 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:55.807584+0000", 2024-04-26T04:32:58.423 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 2.535613713, 2024-04-26T04:32:58.423 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50635941100000004, 2024-04-26T04:32:58.423 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.423 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.423 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:58.423 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.423 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.423 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.423 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 621 2024-04-26T04:32:58.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.423 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807584+0000", 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807584+0000", 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807586+0000", 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.466e-06 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.424 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.425 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.425 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807595+0000", 2024-04-26T04:32:58.425 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.064e-06 2024-04-26T04:32:58.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.425 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.425 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807616+0000", 2024-04-26T04:32:58.425 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1379e-05 2024-04-26T04:32:58.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.425 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.425 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907810+0000", 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10019436800000001 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.009320+0000", 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101509213 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.009472+0000", 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015200399999999999 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.426 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.313944+0000", 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30447191699999998 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:622 3.as0 3:57b6ca82:::smithi17829308-14:b [read 0~229376,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:55.907855+0000", 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 2.4353429339999999, 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.806439982, 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.427 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 622 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907855+0000", 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907855+0000", 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907857+0000", 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2000000000000001e-06 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907869+0000", 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.19e-05 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907889+0000", 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9643000000000001e-05 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907901+0000", 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2289999999999999e-05 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.210837+0000", 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30293618300000003 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.211009+0000", 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017150999999999999 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.714295+0000", 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50328625599999999 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpReadReply(3.as0 60/30 ECSubReadReply(tid=45, attrs_read=0))", 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:56.010127+0000", 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 2.3330705529999998, 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30288447899999998, 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.431 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.010127+0000", 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.010127+0000", 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.432 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.010127+0000", 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.010127+0000", 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.010155+0000", 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7444e-05 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.433 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.010164+0000", 2024-04-26T04:32:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.0769999999999997e-06 2024-04-26T04:32:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.312455+0000", 2024-04-26T04:32:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30229094899999998 2024-04-26T04:32:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.313012+0000", 2024-04-26T04:32:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00055700900000000002 2024-04-26T04:32:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:645 3.as0 3:51a91c23:::smithi17829308-18:head [copy-from ver 49] snapc f=[f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:57.194710+0000", 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 1.1484882430000001, 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.517523497, 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.435 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 645 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194710+0000", 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194710+0000", 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.436 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194714+0000", 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.07e-06 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194727+0000", 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3593e-05 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194762+0000", 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.5086000000000003e-05 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.437 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.438 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194782+0000", 2024-04-26T04:32:58.438 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9349000000000001e-05 2024-04-26T04:32:58.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.438 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.438 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194905+0000", 2024-04-26T04:32:58.438 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000123497 2024-04-26T04:32:58.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.438 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:32:58.438 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194923+0000", 2024-04-26T04:32:58.438 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7377e-05 2024-04-26T04:32:58.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.298801+0000", 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10387803 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.298959+0000", 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000158484 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.509187+0000", 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21022799 2024-04-26T04:32:58.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.440 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.440 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.611520+0000", 2024-04-26T04:32:58.440 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.102332901 2024-04-26T04:32:58.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.440 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:58.440 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.712044+0000", 2024-04-26T04:32:58.440 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10052366 2024-04-26T04:32:58.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.440 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.440 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.712233+0000", 2024-04-26T04:32:58.441 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018945999999999999 2024-04-26T04:32:58.441 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.441 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.441 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.441 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:646 3.as0 3:51a91c23:::smithi17829308-18:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:32:58.441 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:57.194821+0000", 2024-04-26T04:32:58.441 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 1.148376568, 2024-04-26T04:32:58.441 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.416436949, 2024-04-26T04:32:58.441 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.441 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.442 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:58.442 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.442 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.442 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.442 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 646 2024-04-26T04:32:58.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.442 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.442 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.442 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194821+0000", 2024-04-26T04:32:58.442 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194821+0000", 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194822+0000", 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.04e-06 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194827+0000", 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.8760000000000004e-06 2024-04-26T04:32:58.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194841+0000", 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3446e-05 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194850+0000", 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8740000000000001e-06 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.295147+0000", 2024-04-26T04:32:58.444 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100297979 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.295158+0000", 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1e-05 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.499442+0000", 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20428347899999999 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:32:58.445 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.499538+0000", 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6527999999999995e-05 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.610922+0000", 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.111383597 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.611079+0000", 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0001571 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.446 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.447 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.611258+0000", 2024-04-26T04:32:58.447 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017903 2024-04-26T04:32:58.447 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.447 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.447 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.447 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.447 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.447 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:58.449 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T04:32:58.449 INFO:tasks.rados.rados.0.smithi178.stdout:252: finishing write tid 5 to smithi17829308-33 2024-04-26T04:32:58.449 INFO:tasks.rados.rados.0.smithi178.stdout:252: oid 33 updating version 0 to 34 2024-04-26T04:32:58.449 INFO:tasks.rados.rados.0.smithi178.stdout:252: oid 33 updating version 34 to 37 2024-04-26T04:32:58.449 INFO:tasks.rados.rados.0.smithi178.stdout:252: oid 33 version 37 is already newer than 35 2024-04-26T04:32:58.449 INFO:tasks.rados.rados.0.smithi178.stdout:252: oid 33 version 37 is already newer than 36 2024-04-26T04:32:58.449 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 33 v 37 (ObjNum 107 snap 14 seq_num 107) dirty exists 2024-04-26T04:32:58.449 INFO:tasks.rados.rados.0.smithi178.stdout:252: left oid 33 (ObjNum 107 snap 14 seq_num 107) 2024-04-26T04:32:58.450 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:58.445+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:58.551 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:58.549+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:58.577 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:58.577 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:58.577 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 0 2024-04-26T04:32:58.577 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:58.591 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T04:32:58.607 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T04:32:58.653 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:58.653+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:58.713 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:58.713 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:58.713 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:32:58.713 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:32:58.713 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:58.727 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T04:32:58.753 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:32:58.753+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:32:58.764 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:32:58.764 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:32:58.853 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:58.853 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:32:58.853 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:32:58.853 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:32:58.853 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.853 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 36/30 ECSubWrite(tid=29, reqid=client.4670.0:266, at_version=36'17, trim_to=0'0, roll_forward_to=35'16))", 2024-04-26T04:32:58.853 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.819415+0000", 2024-04-26T04:32:58.853 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 29.030314215000001, 2024-04-26T04:32:58.853 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20185976, 2024-04-26T04:32:58.854 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.854 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.854 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.854 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.854 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.854 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.854 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819415+0000", 2024-04-26T04:32:58.854 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.854 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.854 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.854 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.854 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819415+0000", 2024-04-26T04:32:58.855 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.855 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.855 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819416+0000", 2024-04-26T04:32:58.855 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.6000000000000004e-07 2024-04-26T04:32:58.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.855 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.855 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819506+0000", 2024-04-26T04:32:58.855 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.0537000000000004e-05 2024-04-26T04:32:58.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.856 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.856 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819532+0000", 2024-04-26T04:32:58.856 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5709000000000001e-05 2024-04-26T04:32:58.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.856 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.856 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.856 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.819552+0000", 2024-04-26T04:32:58.856 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0373e-05 2024-04-26T04:32:58.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.856 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.856 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.856 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.019889+0000", 2024-04-26T04:32:58.856 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20033689900000001 2024-04-26T04:32:58.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.857 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.857 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.019905+0000", 2024-04-26T04:32:58.857 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5571999999999999e-05 2024-04-26T04:32:58.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.857 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.857 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.021199+0000", 2024-04-26T04:32:58.857 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001294623 2024-04-26T04:32:58.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.858 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.858 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.021275+0000", 2024-04-26T04:32:58.858 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5487000000000004e-05 2024-04-26T04:32:58.858 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.858 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.858 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.858 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.858 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.as1 36/30 ECSubWrite(tid=18, reqid=client.4670.0:265, at_version=0'0, trim_to=0'0, roll_forward_to=36'10))", 2024-04-26T04:32:58.858 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:30.327162+0000", 2024-04-26T04:32:58.858 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 28.522566801, 2024-04-26T04:32:58.858 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20162941100000001, 2024-04-26T04:32:58.858 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.859 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.859 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.859 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.859 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.859 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.859 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327162+0000", 2024-04-26T04:32:58.859 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.859 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.859 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.859 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327162+0000", 2024-04-26T04:32:58.859 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.860 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.860 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.860 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327165+0000", 2024-04-26T04:32:58.860 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1639999999999999e-06 2024-04-26T04:32:58.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.860 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.860 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.860 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327194+0000", 2024-04-26T04:32:58.860 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9388999999999999e-05 2024-04-26T04:32:58.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.860 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.860 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.861 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.327237+0000", 2024-04-26T04:32:58.861 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3065000000000002e-05 2024-04-26T04:32:58.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.861 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.861 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.427384+0000", 2024-04-26T04:32:58.861 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100147443 2024-04-26T04:32:58.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.861 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.861 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.527653+0000", 2024-04-26T04:32:58.861 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100268697 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.527668+0000", 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4449e-05 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.528724+0000", 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0010562869999999999 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.862 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.528792+0000", 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7916999999999994e-05 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:327 3.ds0 3:b7d7af40:::smithi17829308-48:2 [read 0~802816,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.408060+0000", 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 24.441669082000001, 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21258864, 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:58.863 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 327 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408060+0000", 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408060+0000", 2024-04-26T04:32:58.864 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408061+0000", 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.7999999999999995e-07 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408065+0000", 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.25e-06 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.865 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408074+0000", 2024-04-26T04:32:58.866 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1949999999999992e-06 2024-04-26T04:32:58.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.866 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.866 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.866 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408081+0000", 2024-04-26T04:32:58.866 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.2590000000000003e-06 2024-04-26T04:32:58.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.866 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.866 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.866 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408199+0000", 2024-04-26T04:32:58.866 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000118066 2024-04-26T04:32:58.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.866 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408204+0000", 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0289999999999996e-06 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.513122+0000", 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.104917964 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.513302+0000", 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000179092 2024-04-26T04:32:58.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.620649+0000", 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.107347205 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.bs5 40/30 ECSubWrite(tid=106, reqid=client.4670.0:322, at_version=40'19, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.611673+0000", 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 24.238055835000001, 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20070944199999999, 2024-04-26T04:32:58.868 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.611673+0000", 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.611673+0000", 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.870 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.870 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.611674+0000", 2024-04-26T04:32:58.870 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8100000000000001e-07 2024-04-26T04:32:58.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.870 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.870 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.870 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.611815+0000", 2024-04-26T04:32:58.870 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014082299999999999 2024-04-26T04:32:58.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.870 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.870 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.870 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.611830+0000", 2024-04-26T04:32:58.870 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4555999999999999e-05 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.711943+0000", 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100112942 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.712265+0000", 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000322188 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.712273+0000", 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.0020000000000006e-06 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.812299+0000", 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100025989 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.812383+0000", 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4060999999999994e-05 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:381 3.ds0 3:b7d7af40:::smithi17829308-48:5 [read 0~802816,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.431434+0000", 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 19.418295167, 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40271601499999998, 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 381 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.873 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.874 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.874 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431434+0000", 2024-04-26T04:32:58.874 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.874 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.874 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431434+0000", 2024-04-26T04:32:58.874 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.874 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.874 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431436+0000", 2024-04-26T04:32:58.875 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3140000000000002e-06 2024-04-26T04:32:58.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.875 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.875 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431447+0000", 2024-04-26T04:32:58.875 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0655e-05 2024-04-26T04:32:58.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.875 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.875 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431471+0000", 2024-04-26T04:32:58.875 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4470000000000001e-05 2024-04-26T04:32:58.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431488+0000", 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6129000000000001e-05 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.494163+0000", 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.062675393999999995 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.494174+0000", 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.114e-05 2024-04-26T04:32:58.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.497530+0000", 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0033555439999999998 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.497684+0000", 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015432600000000001 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.834150+0000", 2024-04-26T04:32:58.877 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.33646604299999999 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:382 3.ds0 3:b7d7af40:::smithi17829308-48:5 [read 0~802816,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 19.418216559000001, 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50469882499999996, 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.878 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 382 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.879 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9800000000000003e-07 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431521+0000", 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7449999999999995e-06 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431529+0000", 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.3599999999999996e-06 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531644+0000", 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100114514 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.699490+0000", 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.16784555000000001 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.699653+0000", 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016372199999999999 2024-04-26T04:32:58.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.882 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.882 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.882 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.936211+0000", 2024-04-26T04:32:58.882 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23655823600000001 2024-04-26T04:32:58.882 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.882 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.882 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.882 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.882 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.882 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:383 3.ds0 3:b7d7af40:::smithi17829308-48:5 [read 0~802816,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:32:58.882 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.531687+0000", 2024-04-26T04:32:58.882 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 19.318042026000001, 2024-04-26T04:32:58.883 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.405754104, 2024-04-26T04:32:58.883 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.883 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.883 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:58.883 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.883 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.883 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.883 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 383 2024-04-26T04:32:58.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.883 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.883 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.883 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531687+0000", 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531687+0000", 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531689+0000", 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.063e-06 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531702+0000", 2024-04-26T04:32:58.884 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3156000000000001e-05 2024-04-26T04:32:58.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.885 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.885 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531733+0000", 2024-04-26T04:32:58.885 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0335999999999999e-05 2024-04-26T04:32:58.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.885 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.885 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531766+0000", 2024-04-26T04:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3623e-05 2024-04-26T04:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.700194+0000", 2024-04-26T04:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.16842744400000001 2024-04-26T04:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.700343+0000", 2024-04-26T04:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000148763 2024-04-26T04:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.887 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.887 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.937441+0000", 2024-04-26T04:32:58.887 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23709871900000001 2024-04-26T04:32:58.887 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.887 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.887 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.887 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.887 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.887 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpReadReply(3.ds0 44/30 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-26T04:32:58.887 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.632626+0000", 2024-04-26T04:32:58.887 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 19.217103051999999, 2024-04-26T04:32:58.887 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20157424099999999, 2024-04-26T04:32:58.888 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.888 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.888 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.888 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.888 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.888 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.888 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.632626+0000", 2024-04-26T04:32:58.888 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.888 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.888 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.888 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.888 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.632626+0000", 2024-04-26T04:32:58.888 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.889 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.889 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.632628+0000", 2024-04-26T04:32:58.889 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6160000000000001e-06 2024-04-26T04:32:58.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.889 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.889 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.632812+0000", 2024-04-26T04:32:58.889 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018405900000000001 2024-04-26T04:32:58.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.889 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.890 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.632841+0000", 2024-04-26T04:32:58.890 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9051999999999999e-05 2024-04-26T04:32:58.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.890 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.890 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.732979+0000", 2024-04-26T04:32:58.890 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100138158 2024-04-26T04:32:58.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.890 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.890 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.833242+0000", 2024-04-26T04:32:58.890 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10026294500000001 2024-04-26T04:32:58.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.834200+0000", 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00095841100000000005 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:424 3.ds0 3:b7d7af40:::smithi17829308-48:head [append 0~720896] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.921237+0000", 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 13.928492576, 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40632559499999998, 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.891 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 424 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.921237+0000", 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.892 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.921237+0000", 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.921239+0000", 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2900000000000001e-06 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922033+0000", 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00079434500000000001 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.893 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.894 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.894 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922059+0000", 2024-04-26T04:32:58.894 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5925999999999999e-05 2024-04-26T04:32:58.894 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.894 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.894 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.894 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922072+0000", 2024-04-26T04:32:58.894 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2943e-05 2024-04-26T04:32:58.894 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.894 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.894 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.894 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922191+0000", 2024-04-26T04:32:58.894 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000118581 2024-04-26T04:32:58.895 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.895 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.895 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.895 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922362+0000", 2024-04-26T04:32:58.895 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017091599999999999 2024-04-26T04:32:58.895 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.895 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.895 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.895 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.924002+0000", 2024-04-26T04:32:58.895 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016405359999999999 2024-04-26T04:32:58.895 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.895 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.895 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.928906+0000", 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0049035720000000001 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327373+0000", 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39846698000000003 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327562+0000", 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018950600000000001 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.896 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.897 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.897 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:425 3.ds0 3:b7d7af40:::smithi17829308-48:head [append 0~49152] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:32:58.897 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.922098+0000", 2024-04-26T04:32:58.897 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 13.927630732000001, 2024-04-26T04:32:58.897 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.406108466, 2024-04-26T04:32:58.897 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.897 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.897 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:58.897 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.897 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.897 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.897 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 425 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922098+0000", 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922098+0000", 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.898 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922100+0000", 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2300000000000001e-06 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922189+0000", 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9623999999999994e-05 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922208+0000", 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8643000000000001e-05 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.899 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922219+0000", 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0699999999999999e-05 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.925187+0000", 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0029683930000000002 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.925371+0000", 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000184382 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.900 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.926426+0000", 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0010545190000000001 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.928909+0000", 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.002482981 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328000+0000", 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.399091539 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.901 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328207+0000", 2024-04-26T04:32:58.902 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000206455 2024-04-26T04:32:58.902 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.902 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.902 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.902 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.902 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.902 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:426 3.ds0 3:b7d7af40:::smithi17829308-48:head [setxattr _header (56) in=7b] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:32:58.902 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.922241+0000", 2024-04-26T04:32:58.902 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 13.927488576, 2024-04-26T04:32:58.902 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40635279400000002, 2024-04-26T04:32:58.902 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.902 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.902 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 426 2024-04-26T04:32:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922241+0000", 2024-04-26T04:32:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.903 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922241+0000", 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922242+0000", 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0470000000000001e-06 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922246+0000", 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5029999999999999e-06 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.904 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922257+0000", 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.059e-05 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922265+0000", 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.8869999999999992e-06 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.927201+0000", 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0049363640000000004 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.905 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.906 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.927353+0000", 2024-04-26T04:32:58.906 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015164000000000001 2024-04-26T04:32:58.906 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.906 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.906 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.906 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.928331+0000", 2024-04-26T04:32:58.906 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00097882500000000001 2024-04-26T04:32:58.906 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.906 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.906 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.906 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.029157+0000", 2024-04-26T04:32:58.906 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100825282 2024-04-26T04:32:58.906 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328408+0000", 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29925099700000002 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328593+0000", 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018565899999999999 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:427 3.ds0 3:b7d7af40:::smithi17829308-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.922276+0000", 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 13.927453469, 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.57447843799999998, 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 427 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.908 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922276+0000", 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922276+0000", 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922277+0000", 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.7000000000000003e-07 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.910 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.910 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922280+0000", 2024-04-26T04:32:58.910 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0589999999999998e-06 2024-04-26T04:32:58.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.910 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.910 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.910 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922293+0000", 2024-04-26T04:32:58.910 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3617e-05 2024-04-26T04:32:58.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.910 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.910 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.910 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922302+0000", 2024-04-26T04:32:58.910 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4300000000000006e-06 2024-04-26T04:32:58.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.911 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.911 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.929362+0000", 2024-04-26T04:32:58.911 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.007060439 2024-04-26T04:32:58.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.911 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:32:58.911 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.929534+0000", 2024-04-26T04:32:58.911 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017232900000000001 2024-04-26T04:32:58.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.329547+0000", 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40001209799999998 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.329786+0000", 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00023956800000000001 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.496754+0000", 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.16696802799999999 2024-04-26T04:32:58.912 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 50/30 ECSubWriteReply(tid=49, last_complete=50'32, committed=1, applied=1))", 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:45.126605+0000", 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 13.723124501999999, 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20061020500000001, 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.913 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.126605+0000", 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.126605+0000", 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.126606+0000", 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.47e-07 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.914 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.126612+0000", 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.3559999999999998e-06 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.126636+0000", 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4193e-05 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.126652+0000", 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6252999999999998e-05 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.915 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327182+0000", 2024-04-26T04:32:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20052986 2024-04-26T04:32:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327215+0000", 2024-04-26T04:32:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2696e-05 2024-04-26T04:32:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.916 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 50/30 ECSubWriteReply(tid=48, last_complete=50'31, committed=1, applied=1))", 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:45.127432+0000", 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 13.722297623999999, 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20080141000000001, 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127432+0000", 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.917 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127432+0000", 2024-04-26T04:32:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127432+0000", 2024-04-26T04:32:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.0999999999999998e-07 2024-04-26T04:32:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127437+0000", 2024-04-26T04:32:58.918 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.386e-06 2024-04-26T04:32:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127452+0000", 2024-04-26T04:32:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5106000000000001e-05 2024-04-26T04:32:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127461+0000", 2024-04-26T04:32:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.0829999999999993e-06 2024-04-26T04:32:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.919 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327699+0000", 2024-04-26T04:32:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.200237834 2024-04-26T04:32:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328233+0000", 2024-04-26T04:32:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000534291 2024-04-26T04:32:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.920 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 50/30 ECSubWriteReply(tid=49, last_complete=50'32, committed=1, applied=1))", 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:45.127943+0000", 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 13.72178605, 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20148519600000001, 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127943+0000", 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.921 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127943+0000", 2024-04-26T04:32:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127944+0000", 2024-04-26T04:32:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3540000000000001e-06 2024-04-26T04:32:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127953+0000", 2024-04-26T04:32:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.0360000000000006e-06 2024-04-26T04:32:58.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127974+0000", 2024-04-26T04:32:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1396999999999999e-05 2024-04-26T04:32:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.127986+0000", 2024-04-26T04:32:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2566e-05 2024-04-26T04:32:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.923 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328290+0000", 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.200303125 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.329428+0000", 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0011387179999999999 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 56/30 ECSubWrite(tid=60, reqid=client.4670.0:478, at_version=56'42, trim_to=0'0, roll_forward_to=50'39))", 2024-04-26T04:32:58.924 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.321773+0000", 2024-04-26T04:32:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 8.5279566080000002, 2024-04-26T04:32:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50217973100000002, 2024-04-26T04:32:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321773+0000", 2024-04-26T04:32:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.925 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321773+0000", 2024-04-26T04:32:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321773+0000", 2024-04-26T04:32:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4399999999999999e-07 2024-04-26T04:32:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321891+0000", 2024-04-26T04:32:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011750499999999999 2024-04-26T04:32:58.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321903+0000", 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.255e-05 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.421985+0000", 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100081395 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.522172+0000", 2024-04-26T04:32:58.927 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10018666 2024-04-26T04:32:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.522181+0000", 2024-04-26T04:32:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8349999999999999e-06 2024-04-26T04:32:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823898+0000", 2024-04-26T04:32:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30171669200000001 2024-04-26T04:32:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.928 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823952+0000", 2024-04-26T04:32:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4249999999999997e-05 2024-04-26T04:32:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 56/30 ECSubWrite(tid=61, reqid=client.4670.0:479, at_version=56'43, trim_to=0'0, roll_forward_to=50'39))", 2024-04-26T04:32:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.422026+0000", 2024-04-26T04:32:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 8.4277033350000004, 2024-04-26T04:32:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40196675300000001, 2024-04-26T04:32:58.929 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422026+0000", 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422026+0000", 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422026+0000", 2024-04-26T04:32:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.6599999999999996e-07 2024-04-26T04:32:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422072+0000", 2024-04-26T04:32:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5596000000000002e-05 2024-04-26T04:32:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422086+0000", 2024-04-26T04:32:58.931 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3577000000000001e-05 2024-04-26T04:32:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422094+0000", 2024-04-26T04:32:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.6319999999999997e-06 2024-04-26T04:32:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.622832+0000", 2024-04-26T04:32:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20073786099999999 2024-04-26T04:32:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.932 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.622864+0000", 2024-04-26T04:32:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1495000000000003e-05 2024-04-26T04:32:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823965+0000", 2024-04-26T04:32:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20110186299999999 2024-04-26T04:32:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823993+0000", 2024-04-26T04:32:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7163000000000001e-05 2024-04-26T04:32:58.933 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 56/30 ECSubWrite(tid=62, reqid=client.4670.0:480, at_version=56'44, trim_to=0'0, roll_forward_to=50'39))", 2024-04-26T04:32:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.422105+0000", 2024-04-26T04:32:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 8.4276237910000003, 2024-04-26T04:32:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40191106599999998, 2024-04-26T04:32:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.934 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422105+0000", 2024-04-26T04:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422105+0000", 2024-04-26T04:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422106+0000", 2024-04-26T04:32:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1399999999999998e-07 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422115+0000", 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.5319999999999995e-06 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422130+0000", 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4474999999999999e-05 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.936 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422136+0000", 2024-04-26T04:32:58.937 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7039999999999997e-06 2024-04-26T04:32:58.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.937 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.937 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.623647+0000", 2024-04-26T04:32:58.937 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201510879 2024-04-26T04:32:58.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.937 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.937 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.623662+0000", 2024-04-26T04:32:58.937 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5122e-05 2024-04-26T04:32:58.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.938 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.938 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823997+0000", 2024-04-26T04:32:58.938 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.200334135 2024-04-26T04:32:58.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.938 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.938 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.824016+0000", 2024-04-26T04:32:58.938 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9905e-05 2024-04-26T04:32:58.938 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.938 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.938 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.939 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:635 3.ds0 3:b761f772:::smithi17829308-43:head [create,setxattr ]PaibOQQZPXlPQ_TVRVsq`]`lQnrU_adXdhR`ia_ (33) in=40b,setxattr mga[\\pjjlr_f^pjpoNm[SiQinb`p]Uljsr\\PZamg (1614) in=40b,setxattr \\\\nNdqV]P_aoSQ]T][SZi`\\`emqRalsThkWkp]_O (1998) in=40b,setxattr QPRVcO^PYksRnjf^RqtasbqiemoSYpeimc]_kQOo (1505) in=40b,setxattr i`dWfsdYVZUqQmWleQeniob`plYiVWUWSsnsd`\\a (1017) in=40b,setxattr Qn`WN^VrYNfSm_nWkOlO[bZ_[Zci__kgsaZeg[_R (1614) in=40b,setxattr \\]NdaY]dOTpk\\bgtmgifQtgWtaqfd]r`TbktPRjh (19016) in=40b,setxattr NlQZaoabhi (349) in=10b,setxattr QbZ^bqgnd]QnP\\ZjeRjaP[UfZgdm[V^sRaRpnZYY (1963) in=40b,setxattr `]YWO_jgNfUs_SQ`RaqUhXmNY[TtZZbaUj\\NPtY^ (1190) in=40b,setxattr dnfqsNhbe]^ml]ekS[ST`fnk_kpiTUa]Y]ges`rW (17824) in=40b,setxattr RgOUnWhYeYcgnikgiQQWNr_OWg_k`YSaN^XQOdPc (1437) in=40b,setxattr QUmNqZgRZOU]WaWWgl\\pnRhtilqS`cQaScmgUp_e (1438) in=40b,setxattr TfW^Smr_geRfti]efnPdr`iWQUikeXOp`StYqtQd (12280) in=40b,setxattr ZtZ (349) in=3b,setxattr NQYfjd\\ (349) in=7b,setxattr aNaTblXeObdSN[ahZn`j[mcbUaSjeXOnc[dZ^ZZo (910) in=40b,setxattr deb]iS_OUlbmWR_sP^er[TdaV`cSqnnthSotq]YZ (7928) in=40b,setxattr ^s_gVdo]kXUSXb]aqYjbUYbRmcoYksXSWQdVsh (349) in=38b,setxattr h^YTbW\\dli\\^dffQaXRNaZQVkZPONqhhabmhR_[s (1822) in=40b] snapc e=[e,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:32:58.939 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:56.110252+0000", 2024-04-26T04:32:58.939 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 2.739476893, 2024-04-26T04:32:58.939 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40734936900000002, 2024-04-26T04:32:58.939 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.939 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.939 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:32:58.939 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:32:58.939 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:32:58.939 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:32:58.939 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 635 2024-04-26T04:32:58.940 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.940 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.940 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.940 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110252+0000", 2024-04-26T04:32:58.940 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.940 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.940 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.940 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110252+0000", 2024-04-26T04:32:58.940 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.940 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.941 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.941 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110255+0000", 2024-04-26T04:32:58.941 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6560000000000001e-06 2024-04-26T04:32:58.941 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.941 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.941 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.941 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110399+0000", 2024-04-26T04:32:58.941 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000144098 2024-04-26T04:32:58.941 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.941 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.941 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.941 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110437+0000", 2024-04-26T04:32:58.941 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7867000000000002e-05 2024-04-26T04:32:58.942 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.942 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.942 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.942 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110447+0000", 2024-04-26T04:32:58.942 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0183000000000001e-05 2024-04-26T04:32:58.942 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.942 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.942 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.942 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110619+0000", 2024-04-26T04:32:58.942 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017148900000000001 2024-04-26T04:32:58.942 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.942 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.943 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:32:58.943 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110847+0000", 2024-04-26T04:32:58.943 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000228199 2024-04-26T04:32:58.943 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.943 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.943 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:32:58.943 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.112826+0000", 2024-04-26T04:32:58.943 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0019796530000000001 2024-04-26T04:32:58.943 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.943 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.943 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:32:58.943 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.115968+0000", 2024-04-26T04:32:58.943 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.003141657 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.517378+0000", 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.401409617 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.517602+0000", 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022395 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.945 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.945 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 60/30 ECSubWriteReply(tid=59, last_complete=60'38, committed=1, applied=1))", 2024-04-26T04:32:58.945 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:56.299952+0000", 2024-04-26T04:32:58.945 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 2.549777534, 2024-04-26T04:32:58.945 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21846595399999999, 2024-04-26T04:32:58.945 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:32:58.945 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:32:58.945 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:32:58.945 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:32:58.945 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.945 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:32:58.945 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.299952+0000", 2024-04-26T04:32:58.945 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.946 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:32:58.946 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.299952+0000", 2024-04-26T04:32:58.946 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:32:58.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.946 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:32:58.946 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.299954+0000", 2024-04-26T04:32:58.946 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0229999999999999e-06 2024-04-26T04:32:58.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.946 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:32:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.299962+0000", 2024-04-26T04:32:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.6829999999999997e-06 2024-04-26T04:32:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:32:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.299989+0000", 2024-04-26T04:32:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6250000000000001e-05 2024-04-26T04:32:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:32:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.400089+0000", 2024-04-26T04:32:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10009992299999999 2024-04-26T04:32:58.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.517201+0000", 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.117112941 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.518418+0000", 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001216134 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:32:58.948 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:32:58.949 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:58.950 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:32:58.950 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:32:58.951 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:32:58.951 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:32:58.951 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T04:32:58.952 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:32:58.952 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:32:58.953 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:58.949+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:58.958 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:32:58.959 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:32:58.987 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:32:58.987 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:32:59.018 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:32:59.018 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:32:59.047 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:32:59.047 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:32:59.050 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:32:59.046+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:32:59.076 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:32:59.076 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:32:59.082 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:32:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:32:59.082 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 0 2024-04-26T04:32:59.082 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:32:59.096 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T04:32:59.105 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:32:59.105 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:32:59.134 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:32:59.135 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:32:59.151 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:32:59.149+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:32:59.161 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:32:59.161 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:32:59.181 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-04-26T04:32:59.187 INFO:tasks.rados.rados.0.smithi178.stdout:252: done (1 left) 2024-04-26T04:32:59.187 INFO:tasks.rados.rados.0.smithi178.stdout:255: done (0 left) 2024-04-26T04:32:59.187 INFO:tasks.rados.rados.0.smithi178.stdout:256: read oid 15 snap -1 2024-04-26T04:32:59.187 INFO:tasks.rados.rados.0.smithi178.stdout:256: expect (ObjNum 101 snap 13 seq_num 101) 2024-04-26T04:32:59.187 INFO:tasks.rados.rados.0.smithi178.stdout:257: read oid 1 snap -1 2024-04-26T04:32:59.187 INFO:tasks.rados.rados.0.smithi178.stdout:257: expect (ObjNum 86 snap 12 seq_num 86) 2024-04-26T04:32:59.187 INFO:tasks.rados.rados.0.smithi178.stdout:258: read oid 10 snap -1 2024-04-26T04:32:59.187 INFO:tasks.rados.rados.0.smithi178.stdout:258: expect (ObjNum 83 snap 9 seq_num 83) 2024-04-26T04:32:59.187 INFO:tasks.rados.rados.0.smithi178.stdout:259: delete oid 49 current snap is 16 2024-04-26T04:32:59.189 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:32:59.190 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:32:59.220 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:32:59.221 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:32:59.246 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:32:59.246 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:32:59.252 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:32:59.249+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:32:59.262 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:59.262 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:59.270 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:32:59.270 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:32:59.276 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T04:32:59.352 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:32:59.349+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:32:59.398 INFO:tasks.rados.rados.0.smithi178.stdout:259: done (3 left) 2024-04-26T04:32:59.398 INFO:tasks.rados.rados.0.smithi178.stdout:260: delete oid 27 current snap is 16 2024-04-26T04:32:59.399 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:59.399 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:59.414 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T04:32:59.454 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:32:59.453+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:32:59.505 INFO:tasks.rados.rados.0.smithi178.stdout:260: done (3 left) 2024-04-26T04:32:59.505 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 19 current snap is 16 2024-04-26T04:32:59.505 INFO:tasks.rados.rados.0.smithi178.stdout:261: seq_num 110 ranges {0=688128,688128=507904,1196032=655360} 2024-04-26T04:32:59.516 INFO:tasks.rados.rados.0.smithi178.stdout:261: writing smithi17829308-19 from 0 to 688128 tid 1 2024-04-26T04:32:59.524 INFO:tasks.rados.rados.0.smithi178.stdout:261: writing smithi17829308-19 from 688128 to 1196032 tid 2 2024-04-26T04:32:59.536 INFO:tasks.rados.rados.0.smithi178.stdout:261: writing smithi17829308-19 from 1196032 to 1851392 tid 3 2024-04-26T04:32:59.536 INFO:tasks.rados.rados.0.smithi178.stdout:262: delete oid 27 current snap is 16 2024-04-26T04:32:59.542 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:59.542 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:59.555 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:32:59.553+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:32:59.557 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T04:32:59.599 INFO:tasks.rados.rados.0.smithi178.stdout:258: expect (ObjNum 83 snap 9 seq_num 83) 2024-04-26T04:32:59.656 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:59.653+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:59.680 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:59.680 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:59.694 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T04:32:59.701 INFO:tasks.rados.rados.0.smithi178.stdout:257: expect (ObjNum 86 snap 12 seq_num 86) 2024-04-26T04:32:59.751 INFO:tasks.rados.rados.0.smithi178.stdout:257: done (4 left) 2024-04-26T04:32:59.751 INFO:tasks.rados.rados.0.smithi178.stdout:258: done (3 left) 2024-04-26T04:32:59.751 INFO:tasks.rados.rados.0.smithi178.stdout:262: done (2 left) 2024-04-26T04:32:59.751 INFO:tasks.rados.rados.0.smithi178.stdout:263: snap_remove snap 13 2024-04-26T04:32:59.756 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:32:59.754+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:32:59.819 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:59.820 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:59.833 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T04:32:59.879 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:32:59.878+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:32:59.959 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:32:59.957+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:32:59.969 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:32:59.969 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:32:59.984 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T04:33:00.059 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:00.058+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:00.146 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:00.146 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:00.160 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T04:33:00.161 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:00.157+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:00.200 INFO:tasks.rados.rados.0.smithi178.stdout:263: done (2 left) 2024-04-26T04:33:00.201 INFO:tasks.rados.rados.0.smithi178.stdout:264: delete oid 10 current snap is 16 2024-04-26T04:33:00.201 INFO:tasks.rados.rados.0.smithi178.stdout:256: expect (ObjNum 101 snap 13 seq_num 101) 2024-04-26T04:33:00.261 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:00.261+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:00.291 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:00.291 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:00.307 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T04:33:00.326 INFO:tasks.rados.rados.0.smithi178.stdout:261: finishing write tid 1 to smithi17829308-19 2024-04-26T04:33:00.326 INFO:tasks.rados.rados.0.smithi178.stdout:261: finishing write tid 2 to smithi17829308-19 2024-04-26T04:33:00.326 INFO:tasks.rados.rados.0.smithi178.stdout:261: finishing write tid 3 to smithi17829308-19 2024-04-26T04:33:00.326 INFO:tasks.rados.rados.0.smithi178.stdout:261: finishing write tid 4 to smithi17829308-19 2024-04-26T04:33:00.397 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:00.397+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:00.435 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [2] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T04:33:00.435 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T04:33:00.436 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-04-26T04:33:00.436 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:33:00.464 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:00.462+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:00.479 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:00.479 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:00.495 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T04:33:00.565 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:00.560+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:00.609 INFO:tasks.rados.rados.0.smithi178.stdout:256: done (2 left) 2024-04-26T04:33:00.610 INFO:tasks.rados.rados.0.smithi178.stdout:264: done (1 left) 2024-04-26T04:33:00.610 INFO:tasks.rados.rados.0.smithi178.stdout:265: copy_from oid 5 from oid 27 current snap is 16 2024-04-26T04:33:00.610 INFO:tasks.rados.rados.0.smithi178.stdout:266: rollback oid 17 current snap is 16 2024-04-26T04:33:00.610 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 17 to 15 2024-04-26T04:33:00.610 INFO:tasks.rados.rados.0.smithi178.stdout:267: setattr oid 3 current snap is 16 2024-04-26T04:33:00.614 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 21 current snap is 16 2024-04-26T04:33:00.614 INFO:tasks.rados.rados.0.smithi178.stdout:268: seq_num 113 ranges {1310720=737280,2048000=557056,2605056=606208} 2024-04-26T04:33:00.647 INFO:tasks.rados.rados.0.smithi178.stdout:268: writing smithi17829308-21 from 1310720 to 2048000 tid 1 2024-04-26T04:33:00.656 INFO:tasks.rados.rados.0.smithi178.stdout:268: writing smithi17829308-21 from 2048000 to 2605056 tid 2 2024-04-26T04:33:00.658 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:00.658 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:00.664 INFO:tasks.rados.rados.0.smithi178.stdout:268: writing smithi17829308-21 from 2605056 to 3211264 tid 3 2024-04-26T04:33:00.664 INFO:tasks.rados.rados.0.smithi178.stdout:269: read oid 46 snap 12 2024-04-26T04:33:00.665 INFO:tasks.rados.rados.0.smithi178.stdout:269: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-26T04:33:00.665 INFO:tasks.rados.rados.0.smithi178.stdout:270: rollback oid 15 current snap is 16 2024-04-26T04:33:00.665 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 15 to 14 2024-04-26T04:33:00.665 INFO:tasks.rados.rados.0.smithi178.stdout:271: read oid 42 snap 9 2024-04-26T04:33:00.665 INFO:tasks.rados.rados.0.smithi178.stdout:271: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:33:00.665 INFO:tasks.rados.rados.0.smithi178.stdout:272: setattr oid 20 current snap is 16 2024-04-26T04:33:00.666 INFO:tasks.rados.rados.0.smithi178.stdout:261: finishing write tid 5 to smithi17829308-19 2024-04-26T04:33:00.666 INFO:tasks.rados.rados.0.smithi178.stdout:261: oid 19 updating version 0 to 34 2024-04-26T04:33:00.666 INFO:tasks.rados.rados.0.smithi178.stdout:261: oid 19 version 34 is already newer than 33 2024-04-26T04:33:00.666 INFO:tasks.rados.rados.0.smithi178.stdout:261: oid 19 updating version 34 to 35 2024-04-26T04:33:00.666 INFO:tasks.rados.rados.0.smithi178.stdout:261: oid 19 updating version 35 to 36 2024-04-26T04:33:00.666 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 19 v 36 (ObjNum 110 snap 16 seq_num 110) dirty exists 2024-04-26T04:33:00.667 INFO:tasks.rados.rados.0.smithi178.stdout:261: left oid 19 (ObjNum 110 snap 16 seq_num 110) 2024-04-26T04:33:00.668 INFO:tasks.rados.rados.0.smithi178.stdout:261: done (8 left) 2024-04-26T04:33:00.668 INFO:tasks.rados.rados.0.smithi178.stdout:273: read oid 8 snap -1 2024-04-26T04:33:00.668 INFO:tasks.rados.rados.0.smithi178.stdout:273: expect (ObjNum 98 snap 12 seq_num 98) 2024-04-26T04:33:00.669 INFO:tasks.rados.rados.0.smithi178.stdout:274: read oid 22 snap 9 2024-04-26T04:33:00.669 INFO:tasks.rados.rados.0.smithi178.stdout:274: expect (ObjNum 21 snap 0 seq_num 21) 2024-04-26T04:33:00.669 INFO:tasks.rados.rados.0.smithi178.stdout:275: delete oid 47 current snap is 16 2024-04-26T04:33:00.669 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:00.665+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:00.672 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T04:33:00.714 INFO:tasks.rados.rados.0.smithi178.stdout:265: finishing copy_from to smithi17829308-5 2024-04-26T04:33:00.714 INFO:tasks.rados.rados.0.smithi178.stdout:265: got expected ENOENT (src dne) 2024-04-26T04:33:00.715 INFO:tasks.rados.rados.0.smithi178.stdout:265: finishing copy_from racing read to smithi17829308-5 2024-04-26T04:33:00.751 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:33:00.751 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:33:00.769 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:00.769+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:00.771 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-26T04:33:00.196510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":60801568,"num_objects":73,"num_object_clones":22,"num_object_copies":501,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":6,"num_read":485,"num_read_kb":122343,"num_write":678,"num_write_kb":77876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":702,"ondisk_log_size":702,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":15,"num_per_pool_osds":15,"num_per_pool_omap_osds":15,"kb":1415577600,"kb_used":566180,"kb_used_data":110960,"kb_used_omap":23,"kb_used_meta":455016,"kb_avail":1415011420,"statfs":{"total":1449551462400,"available":1448971694080,"internally_reserved":0,"allocated":113623040,"data_stored":131633204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23926,"internal_metadata":465937034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,8,16,0,9,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12550144,"num_objects":14,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":54,"num_read_kb":44592,"num_write":87,"num_write_kb":12256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.029879"},"pg_stats":[{"pgid":"3.f","version":"60'56","reported_seq":230,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-26T04:32:58.543636+0000","last_change":"2024-04-26T04:32:52.691383+0000","last_active":"2024-04-26T04:32:58.543636+0000","last_peered":"2024-04-26T04:32:58.543636+0000","last_clean":"2024-04-26T04:32:58.543636+0000","last_became_active":"2024-04-26T04:32:23.378967+0000","last_became_peered":"2024-04-26T04:32:23.378967+0000","last_unstale":"2024-04-26T04:32:58.543636+0000","last_undegraded":"2024-04-26T04:32:58.543636+0000","last_fullsized":"2024-04-26T04:32:58.543636+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:32.230144+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.50271992099999996,"stat_sum":{"num_bytes":3588096,"num_objects":10,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":46,"num_read_kb":5835,"num_write":62,"num_write_kb":6608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,14,13,8,10,0],"acting":[11,1,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"}]},{"pgid":"3.e","version":"58'26","reported_seq":166,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T04:32:53.615338+0000","last_change":"2024-04-26T04:32:52.485628+0000","last_active":"2024-04-26T04:32:53.615338+0000","last_peered":"2024-04-26T04:32:53.615338+0000","last_clean":"2024-04-26T04:32:53.615338+0000","last_became_active":"2024-04-26T04:32:23.387938+0000","last_became_peered":"2024-04-26T04:32:23.387938+0000","last_unstale":"2024-04-26T04:32:53.615338+0000","last_undegraded":"2024-04-26T04:32:53.615338+0000","last_fullsized":"2024-04-26T04:32:53.615338+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.882749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30381129099999998,"stat_sum":{"num_bytes":2736128,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":10,"num_read_kb":5,"num_write":15,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,9,1,11,8,6,12],"acting":[13,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"3.d","version":"60'38","reported_seq":206,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T04:32:59.196590+0000","last_change":"2024-04-26T04:32:52.392808+0000","last_active":"2024-04-26T04:32:59.196590+0000","last_peered":"2024-04-26T04:32:59.196590+0000","last_clean":"2024-04-26T04:32:59.196590+0000","last_became_active":"2024-04-26T04:32:23.392585+0000","last_became_peered":"2024-04-26T04:32:23.392585+0000","last_unstale":"2024-04-26T04:32:59.196590+0000","last_undegraded":"2024-04-26T04:32:59.196590+0000","last_fullsized":"2024-04-26T04:32:59.196590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:43.907554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013730389999999999,"stat_sum":{"num_bytes":3457024,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":32,"num_read_kb":9754,"num_write":41,"num_write_kb":4336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":148,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-26T04:32:58.543579+0000","last_change":"2024-04-26T04:32:52.189456+0000","last_active":"2024-04-26T04:32:58.543579+0000","last_peered":"2024-04-26T04:32:58.543579+0000","last_clean":"2024-04-26T04:32:58.543579+0000","last_became_active":"2024-04-26T04:32:23.577814+0000","last_became_peered":"2024-04-26T04:32:23.577814+0000","last_unstale":"2024-04-26T04:32:58.543579+0000","last_undegraded":"2024-04-26T04:32:58.543579+0000","last_fullsized":"2024-04-26T04:32:58.543579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:27.420392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001391963,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"}]},{"pgid":"3.b","version":"60'34","reported_seq":197,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-26T04:32:56.593835+0000","last_change":"2024-04-26T04:32:52.683006+0000","last_active":"2024-04-26T04:32:56.593835+0000","last_peered":"2024-04-26T04:32:56.593835+0000","last_clean":"2024-04-26T04:32:56.593835+0000","last_became_active":"2024-04-26T04:32:23.220443+0000","last_became_peered":"2024-04-26T04:32:23.220443+0000","last_unstale":"2024-04-26T04:32:56.593835+0000","last_undegraded":"2024-04-26T04:32:56.593835+0000","last_fullsized":"2024-04-26T04:32:56.593835+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:45.337374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.305636776,"stat_sum":{"num_bytes":5210112,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":38,"num_read_kb":19452,"num_write":22,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"}]},{"pgid":"3.a","version":"62'36","reported_seq":193,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T04:32:59.402710+0000","last_change":"2024-04-26T04:32:52.178079+0000","last_active":"2024-04-26T04:32:59.402710+0000","last_peered":"2024-04-26T04:32:59.402710+0000","last_clean":"2024-04-26T04:32:59.402710+0000","last_became_active":"2024-04-26T04:32:23.310795+0000","last_became_peered":"2024-04-26T04:32:23.310795+0000","last_unstale":"2024-04-26T04:32:59.402710+0000","last_undegraded":"2024-04-26T04:32:59.402710+0000","last_fullsized":"2024-04-26T04:32:59.402710+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:42.507917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013824589999999999,"stat_sum":{"num_bytes":688128,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":18,"num_read_kb":1350,"num_write":17,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"}]},{"pgid":"3.9","version":"44'6","reported_seq":154,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T04:32:55.478539+0000","last_change":"2024-04-26T04:32:52.294222+0000","last_active":"2024-04-26T04:32:55.478539+0000","last_peered":"2024-04-26T04:32:55.478539+0000","last_clean":"2024-04-26T04:32:55.478539+0000","last_became_active":"2024-04-26T04:32:23.476584+0000","last_became_peered":"2024-04-26T04:32:23.476584+0000","last_unstale":"2024-04-26T04:32:55.478539+0000","last_undegraded":"2024-04-26T04:32:55.478539+0000","last_fullsized":"2024-04-26T04:32:55.478539+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:45.794402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012621519999999999,"stat_sum":{"num_bytes":458752,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":2755,"num_write":22,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"3.8","version":"59'66","reported_seq":223,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T04:32:55.488548+0000","last_change":"2024-04-26T04:32:52.387942+0000","last_active":"2024-04-26T04:32:55.488548+0000","last_peered":"2024-04-26T04:32:55.488548+0000","last_clean":"2024-04-26T04:32:55.488548+0000","last_became_active":"2024-04-26T04:32:23.487565+0000","last_became_peered":"2024-04-26T04:32:23.487565+0000","last_unstale":"2024-04-26T04:32:55.488548+0000","last_undegraded":"2024-04-26T04:32:55.488548+0000","last_fullsized":"2024-04-26T04:32:55.488548+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:51.655035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001715706,"stat_sum":{"num_bytes":5013504,"num_objects":9,"num_object_clones":2,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":37,"num_read_kb":11486,"num_write":70,"num_write_kb":5520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,12,1,14,15,6],"acting":[5,0,12,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"1.0","version":"32'157","reported_seq":249,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T04:32:59.402620+0000","last_change":"2024-04-26T04:32:56.378977+0000","last_active":"2024-04-26T04:32:59.402620+0000","last_peered":"2024-04-26T04:32:59.402620+0000","last_clean":"2024-04-26T04:32:59.402620+0000","last_became_active":"2024-04-26T04:32:56.378173+0000","last_became_peered":"2024-04-26T04:32:56.378173+0000","last_unstale":"2024-04-26T04:32:59.402620+0000","last_undegraded":"2024-04-26T04:32:59.402620+0000","last_fullsized":"2024-04-26T04:32:59.402620+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:32:08.365921+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:25.422212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,13],"acting":[3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":78,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T04:32:55.489165+0000","last_change":"2024-04-26T04:32:12.505010+0000","last_active":"2024-04-26T04:32:55.489165+0000","last_peered":"2024-04-26T04:32:55.489165+0000","last_clean":"2024-04-26T04:32:55.489165+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:32:55.489165+0000","last_undegraded":"2024-04-26T04:32:55.489165+0000","last_fullsized":"2024-04-26T04:32:55.489165+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.374592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"44'12","reported_seq":100,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T04:32:57.470280+0000","last_change":"2024-04-26T04:32:56.632797+0000","last_active":"2024-04-26T04:32:57.470280+0000","last_peered":"2024-04-26T04:32:57.470280+0000","last_clean":"2024-04-26T04:32:57.470280+0000","last_became_active":"2024-04-26T04:32:56.531093+0000","last_became_peered":"2024-04-26T04:32:56.531093+0000","last_unstale":"2024-04-26T04:32:57.470280+0000","last_undegraded":"2024-04-26T04:32:57.470280+0000","last_fullsized":"2024-04-26T04:32:57.470280+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:48.440181+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0011542189999999999,"stat_sum":{"num_bytes":360448,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":2117,"num_write":8,"num_write_kb":1584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10,3,6,15,1,11],"acting":[13,10,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T04:32:59.402463+0000","last_change":"2024-04-26T04:32:12.603501+0000","last_active":"2024-04-26T04:32:59.402463+0000","last_peered":"2024-04-26T04:32:59.402463+0000","last_clean":"2024-04-26T04:32:59.402463+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:32:59.402463+0000","last_undegraded":"2024-04-26T04:32:59.402463+0000","last_fullsized":"2024-04-26T04:32:59.402463+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.945207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"58'67","reported_seq":222,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-26T04:32:56.500432+0000","last_change":"2024-04-26T04:32:52.599926+0000","last_active":"2024-04-26T04:32:56.500432+0000","last_peered":"2024-04-26T04:32:56.500432+0000","last_clean":"2024-04-26T04:32:56.500432+0000","last_became_active":"2024-04-26T04:32:23.399115+0000","last_became_peered":"2024-04-26T04:32:23.399115+0000","last_unstale":"2024-04-26T04:32:56.500432+0000","last_undegraded":"2024-04-26T04:32:56.500432+0000","last_fullsized":"2024-04-26T04:32:56.500432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:30.296021+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.32055450699999999,"stat_sum":{"num_bytes":11075584,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":21328,"num_write":47,"num_write_kb":13600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,12,13],"acting":[4,11,9,5,10,12,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":3,"reported_epoch":60,"state":"peering","last_fresh":"2024-04-26T04:32:55.412606+0000","last_change":"2024-04-26T04:32:55.412606+0000","last_active":"2024-04-26T04:32:11.466714+0000","last_peered":"2024-04-26T04:32:11.466714+0000","last_clean":"2024-04-26T04:32:11.466714+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T04:32:55.412606+0000","last_undegraded":"2024-04-26T04:32:55.412606+0000","last_fullsized":"2024-04-26T04:32:55.412606+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,13],"acting":[5,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"59'11","reported_seq":156,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T04:32:55.489672+0000","last_change":"2024-04-26T04:32:52.194059+0000","last_active":"2024-04-26T04:32:55.489672+0000","last_peered":"2024-04-26T04:32:55.489672+0000","last_clean":"2024-04-26T04:32:55.489672+0000","last_became_active":"2024-04-26T04:32:23.386975+0000","last_became_peered":"2024-04-26T04:32:23.386975+0000","last_unstale":"2024-04-26T04:32:55.489672+0000","last_undegraded":"2024-04-26T04:32:55.489672+0000","last_fullsized":"2024-04-26T04:32:55.489672+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:31.109133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258243,"stat_sum":{"num_bytes":1654784,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":8,"num_write_kb":1744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,11,15,1,5,6,4],"acting":[14,11,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"2.2","version":"0'0","reported_seq":85,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-04-26T04:32:59.402899+0000","last_change":"2024-04-26T04:32:12.798464+0000","last_active":"2024-04-26T04:32:59.402899+0000","last_peered":"2024-04-26T04:32:59.402899+0000","last_clean":"2024-04-26T04:32:59.402899+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:32:59.402899+0000","last_undegraded":"2024-04-26T04:32:59.402899+0000","last_fullsized":"2024-04-26T04:32:59.402899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.684262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":140,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T04:32:55.415135+0000","last_change":"2024-04-26T04:32:52.186930+0000","last_active":"2024-04-26T04:32:55.415135+0000","last_peered":"2024-04-26T04:32:55.415135+0000","last_clean":"2024-04-26T04:32:55.415135+0000","last_became_active":"2024-04-26T04:32:23.387136+0000","last_became_peered":"2024-04-26T04:32:23.387136+0000","last_unstale":"2024-04-26T04:32:55.415135+0000","last_undegraded":"2024-04-26T04:32:55.415135+0000","last_fullsized":"2024-04-26T04:32:55.415135+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:42.293393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049429440000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,6,5,3],"acting":[13,7,4,15,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"6","length":"1"},{"start":"9","length":"2"},{"start":"c","length":"2"}]},{"pgid":"3.5","version":"60'50","reported_seq":213,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T04:32:57.498899+0000","last_change":"2024-04-26T04:32:52.391281+0000","last_active":"2024-04-26T04:32:57.498899+0000","last_peered":"2024-04-26T04:32:57.498899+0000","last_clean":"2024-04-26T04:32:57.498899+0000","last_became_active":"2024-04-26T04:32:23.295127+0000","last_became_peered":"2024-04-26T04:32:23.295127+0000","last_unstale":"2024-04-26T04:32:57.498899+0000","last_undegraded":"2024-04-26T04:32:57.498899+0000","last_fullsized":"2024-04-26T04:32:57.498899+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:47.386355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20557355699999999,"stat_sum":{"num_bytes":4964352,"num_objects":7,"num_object_clones":2,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":31,"num_read_kb":12588,"num_write":47,"num_write_kb":5712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":78,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-26T04:32:56.395953+0000","last_change":"2024-04-26T04:32:13.108823+0000","last_active":"2024-04-26T04:32:56.395953+0000","last_peered":"2024-04-26T04:32:56.395953+0000","last_clean":"2024-04-26T04:32:56.395953+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:32:56.395953+0000","last_undegraded":"2024-04-26T04:32:56.395953+0000","last_fullsized":"2024-04-26T04:32:56.395953+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"60'59","reported_seq":220,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T04:32:57.468629+0000","last_change":"2024-04-26T04:32:52.391430+0000","last_active":"2024-04-26T04:32:57.468629+0000","last_peered":"2024-04-26T04:32:57.468629+0000","last_clean":"2024-04-26T04:32:57.468629+0000","last_became_active":"2024-04-26T04:32:23.290645+0000","last_became_peered":"2024-04-26T04:32:23.290645+0000","last_unstale":"2024-04-26T04:32:57.468629+0000","last_undegraded":"2024-04-26T04:32:57.468629+0000","last_fullsized":"2024-04-26T04:32:57.468629+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:46.176717+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.205529189,"stat_sum":{"num_bytes":7405568,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":6395,"num_write":30,"num_write_kb":8112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,4,1,0,14,13],"acting":[8,9,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":75,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T04:32:53.739874+0000","last_change":"2024-04-26T04:32:12.504400+0000","last_active":"2024-04-26T04:32:53.739874+0000","last_peered":"2024-04-26T04:32:53.739874+0000","last_clean":"2024-04-26T04:32:53.739874+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:32:53.739874+0000","last_undegraded":"2024-04-26T04:32:53.739874+0000","last_fullsized":"2024-04-26T04:32:53.739874+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.381954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"60'48","reported_seq":211,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-26T04:32:56.894720+0000","last_change":"2024-04-26T04:32:52.496520+0000","last_active":"2024-04-26T04:32:56.894720+0000","last_peered":"2024-04-26T04:32:56.894720+0000","last_clean":"2024-04-26T04:32:56.894720+0000","last_became_active":"2024-04-26T04:32:23.477983+0000","last_became_peered":"2024-04-26T04:32:23.477983+0000","last_unstale":"2024-04-26T04:32:56.894720+0000","last_undegraded":"2024-04-26T04:32:56.894720+0000","last_fullsized":"2024-04-26T04:32:56.894720+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:43.804710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.217132355,"stat_sum":{"num_bytes":7634944,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":19021,"num_write":26,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":81,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T04:32:57.468568+0000","last_change":"2024-04-26T04:32:12.697838+0000","last_active":"2024-04-26T04:32:57.468568+0000","last_peered":"2024-04-26T04:32:57.468568+0000","last_clean":"2024-04-26T04:32:57.468568+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:32:57.468568+0000","last_undegraded":"2024-04-26T04:32:57.468568+0000","last_fullsized":"2024-04-26T04:32:57.468568+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.961816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"62'36","reported_seq":192,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T04:32:57.703690+0000","last_change":"2024-04-26T04:32:52.887436+0000","last_active":"2024-04-26T04:32:57.703690+0000","last_peered":"2024-04-26T04:32:57.703690+0000","last_clean":"2024-04-26T04:32:57.703690+0000","last_became_active":"2024-04-26T04:32:23.383105+0000","last_became_peered":"2024-04-26T04:32:23.383105+0000","last_unstale":"2024-04-26T04:32:57.703690+0000","last_undegraded":"2024-04-26T04:32:57.703690+0000","last_fullsized":"2024-04-26T04:32:57.703690+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:39.542275+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.70795748000000003,"stat_sum":{"num_bytes":5963776,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":10137,"num_write":60,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"a","length":"1"},{"start":"c","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":81,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-04-26T04:32:57.398937+0000","last_change":"2024-04-26T04:32:12.803651+0000","last_active":"2024-04-26T04:32:57.398937+0000","last_peered":"2024-04-26T04:32:57.398937+0000","last_clean":"2024-04-26T04:32:57.398937+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:32:57.398937+0000","last_undegraded":"2024-04-26T04:32:57.398937+0000","last_fullsized":"2024-04-26T04:32:57.398937+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:26.614652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":60211200,"num_objects":71,"num_object_clones":22,"num_object_copies":497,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":6,"num_read":349,"num_read_kb":122225,"num_write":475,"num_write_kb":74416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":100970496,"data_stored":124780544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":545,"ondisk_log_size":545,"up":112,"acting":112,"num_store_stats":15},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1536000,"data_stored":1525344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247821,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40092,"kb_used_data":8916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331748,"statfs":{"total":96636764160,"available":96595709952,"internally_reserved":0,"allocated":9129984,"data_stored":10494738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247822,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36860,"kb_used_data":5680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334980,"statfs":{"total":96636764160,"available":96599019520,"internally_reserved":0,"allocated":5816320,"data_stored":5710610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247821,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40412,"kb_used_data":9232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331428,"statfs":{"total":96636764160,"available":96595382272,"internally_reserved":0,"allocated":9453568,"data_stored":11232562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313229,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43128,"kb_used_data":11940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328712,"statfs":{"total":96636764160,"available":96592601088,"internally_reserved":0,"allocated":12226560,"data_stored":14580669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313229,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37940,"kb_used_data":10924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333900,"statfs":{"total":96636764160,"available":96597913600,"internally_reserved":0,"allocated":11186176,"data_stored":13607751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,6,13,0,8,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313229,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35616,"kb_used_data":8600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336224,"statfs":{"total":96636764160,"available":96600293376,"internally_reserved":0,"allocated":8806400,"data_stored":9708359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313229,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":5424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":5554176,"data_stored":6779719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,3,4,5,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313229,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":6224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":6373376,"data_stored":8285117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":98784247822,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280525,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29604,"kb_used_data":2580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342236,"statfs":{"total":96636764160,"available":96606449664,"internally_reserved":0,"allocated":2641920,"data_stored":2393683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280525,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43364,"kb_used_data":8020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328476,"statfs":{"total":96636764160,"available":96592359424,"internally_reserved":0,"allocated":8212480,"data_stored":10183731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247821,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38640,"kb_used_data":7460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333200,"statfs":{"total":96636764160,"available":96597196800,"internally_reserved":0,"allocated":7639040,"data_stored":8041234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247822,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37752,"kb_used_data":6572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334088,"statfs":{"total":96636764160,"available":96598106112,"internally_reserved":0,"allocated":6729728,"data_stored":7723936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247820,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37520,"kb_used_data":6340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334320,"statfs":{"total":96636764160,"available":96598343680,"internally_reserved":0,"allocated":6492160,"data_stored":6591250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280525,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41144,"kb_used_data":5800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330696,"statfs":{"total":96636764160,"available":96594632704,"internally_reserved":0,"allocated":5939200,"data_stored":7369779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,0,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247822,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38424,"kb_used_data":7248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333416,"statfs":{"total":96636764160,"available":96597417984,"internally_reserved":0,"allocated":7421952,"data_stored":8930066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11825152,"data_stored":14696448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6836224,"data_stored":7602176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1241088,"data_stored":1384448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10350592,"data_stored":13148160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5910528,"data_stored":7274496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6103040,"data_stored":6549504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5070848,"data_stored":6889472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":7970816,"data_stored":9248768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8327168,"data_stored":10055680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4718592,"data_stored":6320128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":7344128,"data_stored":9703424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":6619136,"data_stored":8556544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":8118272,"data_stored":10264576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":5013504,"data_stored":5271552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":5521408,"data_stored":7815168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:33:00.773 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 16 2024-04-26T04:33:00.851 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:00.852 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:00.867 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T04:33:00.870 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:00.866+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:00.972 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:00.968+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:00.972 INFO:tasks.rados.rados.0.smithi178.stdout:269: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-26T04:33:01.016 INFO:tasks.rados.rados.0.smithi178.stdout:274: expect (ObjNum 21 snap 0 seq_num 21) 2024-04-26T04:33:01.037 INFO:tasks.rados.rados.0.smithi178.stdout:266: finishing rollback tid 0 to smithi17829308-17 2024-04-26T04:33:01.040 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:01.040 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:01.055 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T04:33:01.073 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:01.073+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:01.073 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 20 v 40 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-04-26T04:33:01.073 INFO:tasks.rados.rados.0.smithi178.stdout:270: finishing rollback tid 0 to smithi17829308-15 2024-04-26T04:33:01.079 INFO:tasks.rados.rados.0.smithi178.stdout:273: expect (ObjNum 98 snap 12 seq_num 98) 2024-04-26T04:33:01.168 INFO:teuthology.orchestra.run.smithi012.stderr:set pool 3 pgp_num to 16 2024-04-26T04:33:01.170 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 3 v 58 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-26T04:33:01.173 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:01.170+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:01.174 INFO:tasks.rados.rados.0.smithi178.stdout:266: finishing rollback tid 1 to smithi17829308-17 2024-04-26T04:33:01.174 INFO:tasks.rados.rados.0.smithi178.stdout:266: finishing rollback tid 2 to smithi17829308-17 2024-04-26T04:33:01.174 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 17 v 71 (ObjNum 92 snap 12 seq_num 92) dirty exists 2024-04-26T04:33:01.175 INFO:tasks.rados.rados.0.smithi178.stdout:270: finishing rollback tid 1 to smithi17829308-15 2024-04-26T04:33:01.175 INFO:tasks.rados.rados.0.smithi178.stdout:270: finishing rollback tid 2 to smithi17829308-15 2024-04-26T04:33:01.175 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 15 v 40 (ObjNum 101 snap 13 seq_num 101) dirty exists 2024-04-26T04:33:01.175 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:01.175 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:01.176 INFO:tasks.rados.rados.0.smithi178.stdout:265: done (10 left) 2024-04-26T04:33:01.176 INFO:tasks.rados.rados.0.smithi178.stdout:266: done (9 left) 2024-04-26T04:33:01.176 INFO:tasks.rados.rados.0.smithi178.stdout:267: done (8 left) 2024-04-26T04:33:01.176 INFO:tasks.rados.rados.0.smithi178.stdout:269: done (7 left) 2024-04-26T04:33:01.176 INFO:tasks.rados.rados.0.smithi178.stdout:270: done (6 left) 2024-04-26T04:33:01.176 INFO:tasks.rados.rados.0.smithi178.stdout:272: done (5 left) 2024-04-26T04:33:01.176 INFO:tasks.rados.rados.0.smithi178.stdout:273: done (4 left) 2024-04-26T04:33:01.176 INFO:tasks.rados.rados.0.smithi178.stdout:274: done (3 left) 2024-04-26T04:33:01.177 INFO:tasks.rados.rados.0.smithi178.stdout:275: done (2 left) 2024-04-26T04:33:01.177 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 15 current snap is 16 2024-04-26T04:33:01.177 INFO:tasks.rados.rados.0.smithi178.stdout:276: seq_num 115 ranges {3162112=770048,3932160=704512,4636672=458752} 2024-04-26T04:33:01.193 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T04:33:01.209 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-26T04:33:01.222 INFO:tasks.rados.rados.0.smithi178.stdout:276: writing smithi17829308-15 from 3162112 to 3932160 tid 1 2024-04-26T04:33:01.229 INFO:tasks.rados.rados.0.smithi178.stdout:276: writing smithi17829308-15 from 3932160 to 4636672 tid 2 2024-04-26T04:33:01.234 INFO:tasks.rados.rados.0.smithi178.stdout:276: writing smithi17829308-15 from 4636672 to 5095424 tid 3 2024-04-26T04:33:01.234 INFO:tasks.rados.rados.0.smithi178.stdout:277: delete oid 26 current snap is 16 2024-04-26T04:33:01.234 INFO:tasks.rados.rados.0.smithi178.stdout:268: finishing write tid 1 to smithi17829308-21 2024-04-26T04:33:01.234 INFO:tasks.rados.rados.0.smithi178.stdout:268: finishing write tid 2 to smithi17829308-21 2024-04-26T04:33:01.234 INFO:tasks.rados.rados.0.smithi178.stdout:268: finishing write tid 3 to smithi17829308-21 2024-04-26T04:33:01.281 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:01.281+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:01.284 INFO:tasks.rados.rados.0.smithi178.stdout:271: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:33:01.294 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T04:33:01.358 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:01.358 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:01.375 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:01.373+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:01.376 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T04:33:01.380 INFO:tasks.rados.rados.0.smithi178.stdout:268: finishing write tid 4 to smithi17829308-21 2024-04-26T04:33:01.467 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:33:01.467 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:33:01.476 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:01.473+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:01.498 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:01.498 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:01.503 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:33:01.503 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:33:01.512 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T04:33:01.525 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:33:01.526 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:33:01.547 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:33:01.547 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:33:01.565 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:33:01.565 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:33:01.576 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:01.573+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:01.580 INFO:tasks.rados.rados.0.smithi178.stdout:271: done (3 left) 2024-04-26T04:33:01.580 INFO:tasks.rados.rados.0.smithi178.stdout:277: done (2 left) 2024-04-26T04:33:01.580 INFO:tasks.rados.rados.0.smithi178.stdout:278: copy_from oid 27 from oid 9 current snap is 16 2024-04-26T04:33:01.580 INFO:tasks.rados.rados.0.smithi178.stdout:279: read oid 17 snap -1 2024-04-26T04:33:01.580 INFO:tasks.rados.rados.0.smithi178.stdout:279: expect (ObjNum 92 snap 12 seq_num 92) 2024-04-26T04:33:01.580 INFO:tasks.rados.rados.0.smithi178.stdout:280: read oid 42 snap -1 2024-04-26T04:33:01.580 INFO:tasks.rados.rados.0.smithi178.stdout:280: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:33:01.580 INFO:tasks.rados.rados.0.smithi178.stdout:281: copy_from oid 44 from oid 48 current snap is 16 2024-04-26T04:33:01.580 INFO:tasks.rados.rados.0.smithi178.stdout:282: rmattr oid 1 current snap is 16 2024-04-26T04:33:01.580 INFO:tasks.rados.rados.0.smithi178.stdout:282: done (6 left) 2024-04-26T04:33:01.580 INFO:tasks.rados.rados.0.smithi178.stdout:283: read oid 26 snap -1 2024-04-26T04:33:01.581 INFO:tasks.rados.rados.0.smithi178.stdout:283: expect deleted 2024-04-26T04:33:01.581 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 49 current snap is 16 2024-04-26T04:33:01.581 INFO:tasks.rados.rados.0.smithi178.stdout:284: seq_num 117 ranges {0=425984,425984=606208,1032192=540672,1572864=409600} 2024-04-26T04:33:01.584 INFO:tasks.rados.rados.0.smithi178.stdout:284: writing smithi17829308-49 from 0 to 425984 tid 1 2024-04-26T04:33:01.586 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:33:01.587 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:33:01.589 INFO:tasks.rados.rados.0.smithi178.stdout:284: writing smithi17829308-49 from 425984 to 1032192 tid 2 2024-04-26T04:33:01.594 INFO:tasks.rados.rados.0.smithi178.stdout:284: writing smithi17829308-49 from 1032192 to 1572864 tid 3 2024-04-26T04:33:01.598 INFO:tasks.rados.rados.0.smithi178.stdout:284: writing smithi17829308-49 from 1572864 to 1982464 tid 4 2024-04-26T04:33:01.598 INFO:tasks.rados.rados.0.smithi178.stdout:285: delete oid 7 current snap is 16 2024-04-26T04:33:01.598 INFO:tasks.rados.rados.0.smithi178.stdout:281: finishing copy_from to smithi17829308-44 2024-04-26T04:33:01.598 INFO:tasks.rados.rados.0.smithi178.stdout:281: got expected ENOENT (src dne) 2024-04-26T04:33:01.598 INFO:tasks.rados.rados.0.smithi178.stdout:281: finishing copy_from racing read to smithi17829308-44 2024-04-26T04:33:01.608 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:33:01.609 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:33:01.628 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:01.629 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:01.629 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:33:01.630 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:33:01.642 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T04:33:01.650 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:33:01.650 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:33:01.672 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:33:01.672 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:33:01.677 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:01.674+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:01.691 INFO:tasks.rados.rados.0.smithi178.stdout:276: finishing write tid 1 to smithi17829308-15 2024-04-26T04:33:01.692 INFO:tasks.rados.rados.0.smithi178.stdout:276: finishing write tid 2 to smithi17829308-15 2024-04-26T04:33:01.692 INFO:tasks.rados.rados.0.smithi178.stdout:276: finishing write tid 3 to smithi17829308-15 2024-04-26T04:33:01.698 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:33:01.699 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:33:01.722 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:33:01.722 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:33:01.765 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:01.765 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:01.778 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T04:33:01.778 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:33:01.777+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:33:01.782 INFO:tasks.rados.rados.0.smithi178.stdout:268: finishing write tid 5 to smithi17829308-21 2024-04-26T04:33:01.782 INFO:tasks.rados.rados.0.smithi178.stdout:268: oid 21 updating version 0 to 75 2024-04-26T04:33:01.783 INFO:tasks.rados.rados.0.smithi178.stdout:268: oid 21 version 75 is already newer than 74 2024-04-26T04:33:01.783 INFO:tasks.rados.rados.0.smithi178.stdout:268: oid 21 version 75 is already newer than 73 2024-04-26T04:33:01.783 INFO:tasks.rados.rados.0.smithi178.stdout:268: oid 21 updating version 75 to 76 2024-04-26T04:33:01.783 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 21 v 76 (ObjNum 113 snap 16 seq_num 113) dirty exists 2024-04-26T04:33:01.783 INFO:tasks.rados.rados.0.smithi178.stdout:268: left oid 21 (ObjNum 113 snap 16 seq_num 113) 2024-04-26T04:33:01.792 INFO:tasks.rados.rados.0.smithi178.stdout:276: finishing write tid 4 to smithi17829308-15 2024-04-26T04:33:01.804 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:33:01.804 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:33:01.826 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:33:01.826 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:33:01.849 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:33:01.849 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:33:01.879 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:01.877+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:01.911 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:01.911 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:33:01.911 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 0 2024-04-26T04:33:01.911 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:01.913 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:33:01.913 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:33:01.924 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T04:33:01.980 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:01.976+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:01.992 INFO:tasks.rados.rados.0.smithi178.stdout:268: done (8 left) 2024-04-26T04:33:01.992 INFO:tasks.rados.rados.0.smithi178.stdout:281: done (7 left) 2024-04-26T04:33:01.992 INFO:tasks.rados.rados.0.smithi178.stdout:283: done (6 left) 2024-04-26T04:33:01.992 INFO:tasks.rados.rados.0.smithi178.stdout:285: done (5 left) 2024-04-26T04:33:01.993 INFO:tasks.rados.rados.0.smithi178.stdout:286: delete oid 28 current snap is 16 2024-04-26T04:33:02.081 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:02.080+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:02.090 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:02.090 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:33:02.090 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:33:02.090 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:33:02.090 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:02.092 INFO:tasks.rados.rados.0.smithi178.stdout:276: finishing write tid 5 to smithi17829308-15 2024-04-26T04:33:02.092 INFO:tasks.rados.rados.0.smithi178.stdout:276: oid 15 updating version 0 to 46 2024-04-26T04:33:02.092 INFO:tasks.rados.rados.0.smithi178.stdout:276: oid 15 version 46 is already newer than 43 2024-04-26T04:33:02.093 INFO:tasks.rados.rados.0.smithi178.stdout:276: oid 15 version 46 is already newer than 44 2024-04-26T04:33:02.093 INFO:tasks.rados.rados.0.smithi178.stdout:276: oid 15 version 46 is already newer than 45 2024-04-26T04:33:02.093 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 15 v 46 (ObjNum 115 snap 16 seq_num 115) dirty exists 2024-04-26T04:33:02.093 INFO:tasks.rados.rados.0.smithi178.stdout:276: left oid 15 (ObjNum 115 snap 16 seq_num 115) 2024-04-26T04:33:02.093 INFO:tasks.rados.rados.0.smithi178.stdout:279: expect (ObjNum 92 snap 12 seq_num 92) 2024-04-26T04:33:02.104 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T04:33:02.181 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:02.181+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:02.208 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-04-26T04:33:02.230 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:02.230 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:33:02.230 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:33:02.230 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:33:02.230 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.230 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 35/30 ECSubWrite(tid=34, reqid=client.4670.0:202, at_version=35'24, trim_to=0'0, roll_forward_to=35'22))", 2024-04-26T04:33:02.231 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.161613+0000", 2024-04-26T04:33:02.231 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 33.066459578, 2024-04-26T04:33:02.231 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.103291546, 2024-04-26T04:33:02.231 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.231 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.231 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.231 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.231 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.231 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.231 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.161613+0000", 2024-04-26T04:33:02.231 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.231 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.231 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.161613+0000", 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.161615+0000", 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.204e-06 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.162484+0000", 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00086872700000000002 2024-04-26T04:33:02.232 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.162526+0000", 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1791e-05 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.162550+0000", 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4341999999999998e-05 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.262786+0000", 2024-04-26T04:33:02.233 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100236265 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.262801+0000", 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5012e-05 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.264812+0000", 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.002010333 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.234 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.264905+0000", 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.2872000000000001e-05 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 45/30 ECSubWrite(tid=31, reqid=client.4670.0:374, at_version=45'20, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.532047+0000", 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.696026071999999, 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17995593700000001, 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.235 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532047+0000", 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532047+0000", 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532048+0000", 2024-04-26T04:33:02.236 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5960000000000001e-06 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532346+0000", 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00029776800000000001 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532375+0000", 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8915e-05 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.237 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532395+0000", 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9663000000000001e-05 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608323+0000", 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.075928566000000003 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608336+0000", 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3377000000000001e-05 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.238 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.239 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.711932+0000", 2024-04-26T04:33:02.239 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10359542099999999 2024-04-26T04:33:02.239 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.239 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.239 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.239 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.712003+0000", 2024-04-26T04:33:02.239 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0630999999999995e-05 2024-04-26T04:33:02.239 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.239 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.239 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.239 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.239 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.240 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 45/30 ECSubWrite(tid=32, reqid=client.4670.0:375, at_version=45'21, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:33:02.240 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.533950+0000", 2024-04-26T04:33:02.240 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.694122382, 2024-04-26T04:33:02.240 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17809028800000001, 2024-04-26T04:33:02.240 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.240 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.240 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.240 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.240 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.240 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.240 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.533950+0000", 2024-04-26T04:33:02.240 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.240 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.533950+0000", 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.533951+0000", 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1200000000000002e-07 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.534075+0000", 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012397999999999999 2024-04-26T04:33:02.241 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.242 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.242 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.242 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.534089+0000", 2024-04-26T04:33:02.242 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4097000000000001e-05 2024-04-26T04:33:02.242 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.242 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.242 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.242 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.534097+0000", 2024-04-26T04:33:02.242 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.6420000000000007e-06 2024-04-26T04:33:02.242 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.242 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.242 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.242 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.709588+0000", 2024-04-26T04:33:02.243 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17549146800000001 2024-04-26T04:33:02.243 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.243 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.243 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.243 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.709604+0000", 2024-04-26T04:33:02.243 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5400000000000002e-05 2024-04-26T04:33:02.243 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.243 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.243 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.243 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.712011+0000", 2024-04-26T04:33:02.243 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0024075469999999999 2024-04-26T04:33:02.243 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.243 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.244 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.244 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.712041+0000", 2024-04-26T04:33:02.244 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9342000000000001e-05 2024-04-26T04:33:02.244 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.244 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.244 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.244 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.244 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.244 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 45/30 ECSubWrite(tid=33, reqid=client.4670.0:376, at_version=45'22, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:33:02.244 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.535418+0000", 2024-04-26T04:33:02.244 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.692654732000001, 2024-04-26T04:33:02.244 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.177833458, 2024-04-26T04:33:02.244 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.245 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.245 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.245 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.245 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.245 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.245 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535418+0000", 2024-04-26T04:33:02.245 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.245 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.245 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.245 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.245 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535418+0000", 2024-04-26T04:33:02.245 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.246 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.246 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.246 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.246 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535419+0000", 2024-04-26T04:33:02.246 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6400000000000001e-07 2024-04-26T04:33:02.246 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.246 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.246 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.246 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535502+0000", 2024-04-26T04:33:02.246 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.365e-05 2024-04-26T04:33:02.246 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.246 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535517+0000", 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4489e-05 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535525+0000", 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4940000000000003e-06 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.710429+0000", 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17490356100000001 2024-04-26T04:33:02.247 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.710454+0000", 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5038999999999999e-05 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.713205+0000", 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00275055 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.713251+0000", 2024-04-26T04:33:02.248 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.6811000000000003e-05 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 45/30 ECSubWrite(tid=34, reqid=client.4670.0:377, at_version=45'23, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.536571+0000", 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.691501176999999, 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17941110099999999, 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.249 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.250 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536571+0000", 2024-04-26T04:33:02.250 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.250 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.250 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.250 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.250 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536571+0000", 2024-04-26T04:33:02.250 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.250 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.250 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.250 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.250 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536572+0000", 2024-04-26T04:33:02.250 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.0999999999999998e-07 2024-04-26T04:33:02.250 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536589+0000", 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7110000000000001e-05 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536601+0000", 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1368e-05 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536608+0000", 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2490000000000004e-06 2024-04-26T04:33:02.251 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.252 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.252 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.252 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.711524+0000", 2024-04-26T04:33:02.252 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17491638100000001 2024-04-26T04:33:02.252 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.252 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.252 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.252 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.711553+0000", 2024-04-26T04:33:02.252 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9142e-05 2024-04-26T04:33:02.252 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.252 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.252 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.252 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.715942+0000", 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00438895 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.715983+0000", 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.0290999999999998e-05 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 47/30 ECSubWrite(tid=55, reqid=osd.5.0:55, at_version=47'43, trim_to=0'0, roll_forward_to=44'41))", 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:41.211282+0000", 2024-04-26T04:33:02.253 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 21.016790528000001, 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.25277038499999999, 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211282+0000", 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211282+0000", 2024-04-26T04:33:02.254 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211283+0000", 2024-04-26T04:33:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6400000000000002e-07 2024-04-26T04:33:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211337+0000", 2024-04-26T04:33:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.4304e-05 2024-04-26T04:33:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.255 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211361+0000", 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3983000000000001e-05 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211395+0000", 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4255000000000003e-05 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.363290+0000", 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.15189467800000001 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.256 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.363307+0000", 2024-04-26T04:33:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6863000000000001e-05 2024-04-26T04:33:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.463984+0000", 2024-04-26T04:33:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100677144 2024-04-26T04:33:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.464053+0000", 2024-04-26T04:33:02.257 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.8493999999999998e-05 2024-04-26T04:33:02.258 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.258 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.258 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.258 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.258 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.258 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "pg_update_log_missing(3.8s3 epoch 50/30 rep_tid 66 entries 50'54 (0'0) error 3:1d1feae2:::smithi17829308-1:head by client.4670.0:422 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 50'51)", 2024-04-26T04:33:02.258 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:44.585396+0000", 2024-04-26T04:33:02.258 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 17.642676757, 2024-04-26T04:33:02.258 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20079079399999999, 2024-04-26T04:33:02.258 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.258 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.258 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.258 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.259 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.259 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.259 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585396+0000", 2024-04-26T04:33:02.259 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.259 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.259 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.259 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.259 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585396+0000", 2024-04-26T04:33:02.259 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.259 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.259 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.259 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.259 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585398+0000", 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7999999999999999e-06 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585418+0000", 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0358999999999999e-05 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585468+0000", 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.9444999999999998e-05 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.260 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.261 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.685596+0000", 2024-04-26T04:33:02.261 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100128605 2024-04-26T04:33:02.261 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.261 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.261 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.261 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.685857+0000", 2024-04-26T04:33:02.261 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026076799999999998 2024-04-26T04:33:02.261 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.261 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.261 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.261 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.786187+0000", 2024-04-26T04:33:02.261 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100329817 2024-04-26T04:33:02.261 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 52/30 ECSubWrite(tid=54, reqid=client.4670.0:435, at_version=52'43, trim_to=0'0, roll_forward_to=50'41))", 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:46.228501+0000", 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 15.999571635000001, 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10352711000000001, 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.262 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.263 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.228501+0000", 2024-04-26T04:33:02.263 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.263 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.263 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.263 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.263 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.228501+0000", 2024-04-26T04:33:02.263 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.263 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.263 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.263 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.263 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.228516+0000", 2024-04-26T04:33:02.263 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5270000000000001e-05 2024-04-26T04:33:02.263 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.229103+0000", 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00058647599999999995 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.229122+0000", 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9633999999999999e-05 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.229137+0000", 2024-04-26T04:33:02.264 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4929000000000001e-05 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.229262+0000", 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012421900000000001 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.229276+0000", 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4581e-05 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.265 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.331983+0000", 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102707061 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.332028+0000", 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.494e-05 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.fs1 52/30 ECSubWrite(tid=57, reqid=client.4670.0:431, at_version=52'36, trim_to=0'0, roll_forward_to=48'33))", 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:46.434158+0000", 2024-04-26T04:33:02.266 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 15.793914298000001, 2024-04-26T04:33:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20233947599999999, 2024-04-26T04:33:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.434158+0000", 2024-04-26T04:33:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.267 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.434158+0000", 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.434159+0000", 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7300000000000005e-07 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.434425+0000", 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026552800000000001 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.268 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.434448+0000", 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3425e-05 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.534531+0000", 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10008265700000001 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.534635+0000", 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000104326 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.269 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.534644+0000", 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.551e-06 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.636402+0000", 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10175832899999999 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.636498+0000", 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.5886999999999999e-05 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.270 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.6s2 52/30 ECSubWrite(tid=28, reqid=client.4670.0:440, at_version=52'18, trim_to=0'0, roll_forward_to=44'16))", 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:46.535059+0000", 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 15.693014083, 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10348803500000001, 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.535059+0000", 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.271 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.535059+0000", 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.535059+0000", 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2700000000000001e-07 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.535175+0000", 2024-04-26T04:33:02.272 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000116125 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.535188+0000", 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3485999999999999e-05 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.635359+0000", 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100170235 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.635571+0000", 2024-04-26T04:33:02.273 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021249600000000001 2024-04-26T04:33:02.274 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.274 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.274 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.274 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.635601+0000", 2024-04-26T04:33:02.274 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9663e-05 2024-04-26T04:33:02.274 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.274 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.274 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.274 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.638519+0000", 2024-04-26T04:33:02.274 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0029182019999999999 2024-04-26T04:33:02.274 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.274 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.274 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.638547+0000", 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7600999999999999e-05 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es2 60/30 ECSubWrite(tid=213, reqid=client.4670.0:582, at_version=60'28, trim_to=0'0, roll_forward_to=58'25))", 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.714497+0000", 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.5135755800000004, 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10370457600000001, 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.275 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.714497+0000", 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.714497+0000", 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.276 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.714498+0000", 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.0699999999999997e-07 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.714994+0000", 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00049615900000000003 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.715032+0000", 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8442000000000002e-05 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.277 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.815148+0000", 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100116134 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.815252+0000", 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000104147 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.815267+0000", 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4056000000000001e-05 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.278 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.818099+0000", 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0028327199999999999 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.818202+0000", 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010241100000000001 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.fs1 60/30 ECSubWrite(tid=89, reqid=client.4670.0:607, at_version=60'56, trim_to=0'0, roll_forward_to=59'54))", 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.876722+0000", 2024-04-26T04:33:02.279 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.3513507310000001, 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20298210799999999, 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.876722+0000", 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.876722+0000", 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.280 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.876723+0000", 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0599999999999999e-07 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.876913+0000", 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000190669 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.876928+0000", 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.503e-05 2024-04-26T04:33:02.281 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.876938+0000", 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3489999999999998e-06 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.077326+0000", 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20038803099999999 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.077352+0000", 2024-04-26T04:33:02.282 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6533000000000002e-05 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.079616+0000", 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0022639090000000001 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.079704+0000", 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7781000000000005e-05 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.283 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.as2 60/30 ECSubRead(tid=47, to_read={3:57b6ca82:::smithi17829308-14:head=0,57344,0}, subchunks={3:57b6ca82:::smithi17829308-14:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:56.212241+0000", 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.0158312030000003, 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17507404200000001, 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.212241+0000", 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.284 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.212241+0000", 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.212242+0000", 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9999999999999998e-07 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.212254+0000", 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1686e-05 2024-04-26T04:33:02.285 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.212291+0000", 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.7778999999999999e-05 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.212311+0000", 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9726000000000001e-05 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.387083+0000", 2024-04-26T04:33:02.286 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17477169000000001 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.387316+0000", 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000232761 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 60/30 ECSubWrite(tid=96, reqid=client.4670.0:577, at_version=60'75, trim_to=0'0, roll_forward_to=59'66))", 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:56.328863+0000", 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.8992092180000002, 2024-04-26T04:33:02.287 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10316083099999999, 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.328863+0000", 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.328863+0000", 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.288 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.328865+0000", 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5060000000000001e-06 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.328960+0000", 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.5534999999999999e-05 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.329009+0000", 2024-04-26T04:33:02.289 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.8105000000000001e-05 2024-04-26T04:33:02.290 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.290 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.290 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.290 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.429105+0000", 2024-04-26T04:33:02.290 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10009683 2024-04-26T04:33:02.290 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.290 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.290 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.290 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.429319+0000", 2024-04-26T04:33:02.290 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000213551 2024-04-26T04:33:02.290 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.290 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.290 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.429340+0000", 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1418999999999999e-05 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.431930+0000", 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0025897170000000001 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.432024+0000", 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.4167999999999997e-05 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.291 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.292 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.292 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 61/30 ECSubWrite(tid=98, reqid=client.4670.0:624, at_version=61'76, trim_to=0'0, roll_forward_to=60'71))", 2024-04-26T04:33:02.292 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:56.636689+0000", 2024-04-26T04:33:02.292 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.591384025, 2024-04-26T04:33:02.292 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.203428576, 2024-04-26T04:33:02.292 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.292 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.292 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.292 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.292 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.292 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.292 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636689+0000", 2024-04-26T04:33:02.292 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636689+0000", 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636691+0000", 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8840000000000001e-06 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.293 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636916+0000", 2024-04-26T04:33:02.294 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0002252 2024-04-26T04:33:02.294 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.294 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.294 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.294 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636947+0000", 2024-04-26T04:33:02.294 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0973000000000003e-05 2024-04-26T04:33:02.294 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.294 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.294 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.294 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636967+0000", 2024-04-26T04:33:02.294 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0041999999999999e-05 2024-04-26T04:33:02.294 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.294 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.737233+0000", 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10026647800000001 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.737252+0000", 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8508999999999998e-05 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.840059+0000", 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102807022 2024-04-26T04:33:02.295 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.840117+0000", 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.8468000000000003e-05 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 61/30 ECSubWrite(tid=99, reqid=client.4670.0:625, at_version=61'77, trim_to=0'0, roll_forward_to=60'73))", 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:56.637006+0000", 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.591066777, 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20363767699999999, 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.296 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637006+0000", 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637006+0000", 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.297 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637007+0000", 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.063e-06 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637211+0000", 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020410200000000001 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637231+0000", 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9508999999999999e-05 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.298 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637243+0000", 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2490000000000001e-05 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.838505+0000", 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20126216099999999 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.838520+0000", 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.486e-05 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.299 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.840583+0000", 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0020633079999999998 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.840644+0000", 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.0183999999999998e-05 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 61/30 ECSubWrite(tid=100, reqid=client.4670.0:626, at_version=61'78, trim_to=0'0, roll_forward_to=60'73))", 2024-04-26T04:33:02.300 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:56.637270+0000", 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.5908025209999996, 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.203845515, 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637270+0000", 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637270+0000", 2024-04-26T04:33:02.301 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.302 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.302 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.302 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.302 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637271+0000", 2024-04-26T04:33:02.302 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7700000000000004e-07 2024-04-26T04:33:02.302 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.302 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.302 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.302 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637358+0000", 2024-04-26T04:33:02.302 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6990999999999994e-05 2024-04-26T04:33:02.302 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.302 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.302 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637375+0000", 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7125999999999998e-05 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637385+0000", 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.043e-05 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.839422+0000", 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20203700699999999 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.303 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.839438+0000", 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5993000000000001e-05 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.841059+0000", 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016203999999999999 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.841116+0000", 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.6790999999999999e-05 2024-04-26T04:33:02.304 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 61/30 ECSubWrite(tid=101, reqid=client.4670.0:627, at_version=61'79, trim_to=0'0, roll_forward_to=60'73))", 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:56.637402+0000", 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.5906708209999998, 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.204692506, 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.305 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637402+0000", 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637402+0000", 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637402+0000", 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.37e-07 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.306 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637419+0000", 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6555999999999999e-05 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637433+0000", 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.403e-05 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637442+0000", 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.4520000000000003e-06 2024-04-26T04:33:02.307 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.840213+0000", 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20277055499999999 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.840229+0000", 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5752000000000001e-05 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.842033+0000", 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001804625 2024-04-26T04:33:02.308 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.842094+0000", 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.0999000000000003e-05 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.6s2 62/30 ECSubWrite(tid=52, reqid=client.4670.0:642, at_version=62'36, trim_to=0'0, roll_forward_to=60'32))", 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:57.502887+0000", 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.7251859869999997, 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.103020102, 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.309 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.502887+0000", 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.502887+0000", 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.310 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.502925+0000", 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8556000000000002e-05 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.503267+0000", 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00034210099999999998 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.503297+0000", 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9745999999999999e-05 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.311 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.503314+0000", 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6500000000000001e-05 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.603590+0000", 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100276115 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.603620+0000", 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0766000000000003e-05 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.312 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.605836+0000", 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0022158429999999999 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.605907+0000", 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0475000000000005e-05 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs2 66/30 ECSubWrite(tid=257, reqid=client.4670.0:715, at_version=66'39, trim_to=0'0, roll_forward_to=65'38))", 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:01.685166+0000", 2024-04-26T04:33:02.313 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.542906373, 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10326983100000001, 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.685166+0000", 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.685166+0000", 2024-04-26T04:33:02.314 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.685168+0000", 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2669999999999999e-06 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.685543+0000", 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00037592799999999998 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.315 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.685569+0000", 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5196000000000001e-05 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.685589+0000", 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0519000000000001e-05 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.785046+0000", 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.099456618999999996 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.316 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.785059+0000", 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2863e-05 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.788342+0000", 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.003282994 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.788436+0000", 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.4444999999999994e-05 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.317 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.318 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.318 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.318 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:02.321 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T04:33:02.321 INFO:tasks.rados.rados.0.smithi178.stdout:278: finishing copy_from racing read to smithi17829308-27 2024-04-26T04:33:02.321 INFO:tasks.rados.rados.0.smithi178.stdout:284: finishing write tid 1 to smithi17829308-49 2024-04-26T04:33:02.322 INFO:tasks.rados.rados.0.smithi178.stdout:284: finishing write tid 2 to smithi17829308-49 2024-04-26T04:33:02.322 INFO:tasks.rados.rados.0.smithi178.stdout:276: done (5 left) 2024-04-26T04:33:02.322 INFO:tasks.rados.rados.0.smithi178.stdout:279: done (4 left) 2024-04-26T04:33:02.322 INFO:tasks.rados.rados.0.smithi178.stdout:286: done (3 left) 2024-04-26T04:33:02.322 INFO:tasks.rados.rados.0.smithi178.stdout:287: copy_from oid 44 from oid 37 current snap is 16 2024-04-26T04:33:02.322 INFO:tasks.rados.rados.0.smithi178.stdout:288: read oid 31 snap -1 2024-04-26T04:33:02.322 INFO:tasks.rados.rados.0.smithi178.stdout:288: expect (ObjNum 31 snap 0 seq_num 31) 2024-04-26T04:33:02.322 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 26 current snap is 16 2024-04-26T04:33:02.322 INFO:tasks.rados.rados.0.smithi178.stdout:289: seq_num 120 ranges {0=475136,475136=655360,1130496=622592,1753088=278528} 2024-04-26T04:33:02.322 INFO:tasks.rados.rados.0.smithi178.stdout:289: writing smithi17829308-26 from 0 to 475136 tid 1 2024-04-26T04:33:02.322 INFO:tasks.rados.rados.0.smithi178.stdout:289: writing smithi17829308-26 from 475136 to 1130496 tid 2 2024-04-26T04:33:02.323 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:02.316+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:02.324 INFO:tasks.rados.rados.0.smithi178.stdout:289: writing smithi17829308-26 from 1130496 to 1753088 tid 3 2024-04-26T04:33:02.328 INFO:tasks.rados.rados.0.smithi178.stdout:289: writing smithi17829308-26 from 1753088 to 2031616 tid 4 2024-04-26T04:33:02.328 INFO:tasks.rados.rados.0.smithi178.stdout:290: rmattr oid 16 current snap is 16 2024-04-26T04:33:02.328 INFO:tasks.rados.rados.0.smithi178.stdout:290: done (6 left) 2024-04-26T04:33:02.329 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 4 current snap is 16 2024-04-26T04:33:02.329 INFO:tasks.rados.rados.0.smithi178.stdout:291: seq_num 121 ranges {65536=491520,557056=802816,1359872=720896,2080768=32768} 2024-04-26T04:33:02.336 INFO:tasks.rados.rados.0.smithi178.stdout:291: writing smithi17829308-4 from 65536 to 557056 tid 1 2024-04-26T04:33:02.346 INFO:tasks.rados.rados.0.smithi178.stdout:291: writing smithi17829308-4 from 557056 to 1359872 tid 2 2024-04-26T04:33:02.355 INFO:tasks.rados.rados.0.smithi178.stdout:291: writing smithi17829308-4 from 1359872 to 2080768 tid 3 2024-04-26T04:33:02.355 INFO:tasks.rados.rados.0.smithi178.stdout:291: writing smithi17829308-4 from 2080768 to 2113536 tid 4 2024-04-26T04:33:02.355 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 20 current snap is 16 2024-04-26T04:33:02.355 INFO:tasks.rados.rados.0.smithi178.stdout:292: seq_num 122 ranges {327680=507904,835584=425984,1261568=442368,1703936=540672,2244608=147456} 2024-04-26T04:33:02.363 INFO:tasks.rados.rados.0.smithi178.stdout:292: writing smithi17829308-20 from 327680 to 835584 tid 1 2024-04-26T04:33:02.367 INFO:tasks.rados.rados.0.smithi178.stdout:292: writing smithi17829308-20 from 835584 to 1261568 tid 2 2024-04-26T04:33:02.371 INFO:tasks.rados.rados.0.smithi178.stdout:292: writing smithi17829308-20 from 1261568 to 1703936 tid 3 2024-04-26T04:33:02.378 INFO:tasks.rados.rados.0.smithi178.stdout:292: writing smithi17829308-20 from 1703936 to 2244608 tid 4 2024-04-26T04:33:02.379 INFO:tasks.rados.rados.0.smithi178.stdout:292: writing smithi17829308-20 from 2244608 to 2392064 tid 5 2024-04-26T04:33:02.379 INFO:tasks.rados.rados.0.smithi178.stdout:293: copy_from oid 10 from oid 11 current snap is 16 2024-04-26T04:33:02.379 INFO:tasks.rados.rados.0.smithi178.stdout:278: finishing copy_from to smithi17829308-27 2024-04-26T04:33:02.379 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 27 v 44 (ObjNum 87 snap 12 seq_num 87) dirty exists 2024-04-26T04:33:02.379 INFO:tasks.rados.rados.0.smithi178.stdout:278: done (8 left) 2024-04-26T04:33:02.379 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 33 current snap is 16 2024-04-26T04:33:02.379 INFO:tasks.rados.rados.0.smithi178.stdout:294: seq_num 123 ranges {2342912=524288,2867200=573440,3440640=688128,4128768=294912} 2024-04-26T04:33:02.399 INFO:tasks.rados.rados.0.smithi178.stdout:294: writing smithi17829308-33 from 2342912 to 2867200 tid 1 2024-04-26T04:33:02.403 INFO:tasks.rados.rados.0.smithi178.stdout:294: writing smithi17829308-33 from 2867200 to 3440640 tid 2 2024-04-26T04:33:02.408 INFO:tasks.rados.rados.0.smithi178.stdout:294: writing smithi17829308-33 from 3440640 to 4128768 tid 3 2024-04-26T04:33:02.410 INFO:tasks.rados.rados.0.smithi178.stdout:294: writing smithi17829308-33 from 4128768 to 4423680 tid 4 2024-04-26T04:33:02.410 INFO:tasks.rados.rados.0.smithi178.stdout:295: snap_remove snap 9 2024-04-26T04:33:02.420 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:02.418+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:02.493 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:02.493 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:33:02.493 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 0 2024-04-26T04:33:02.493 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:02.508 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T04:33:02.553 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:02.553+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:02.621 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:02.621+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:02.630 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:02.630 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:33:02.630 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:33:02.630 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:33:02.630 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:02.644 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T04:33:02.723 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:02.721+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:02.815 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:02.816 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:33:02.816 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:33:02.816 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:33:02.816 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.816 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:1 3.8s0 3:1d1feae2:::smithi17829308-1:head [append 0~32768,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:02.816 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.200828+0000", 2024-04-26T04:33:02.816 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 37.611268795000001, 2024-04-26T04:33:02.816 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.311358421, 2024-04-26T04:33:02.816 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.816 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.816 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:02.816 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 1 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200828+0000", 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200828+0000", 2024-04-26T04:33:02.817 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200830+0000", 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.57e-06 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200962+0000", 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000132115 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.818 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.200985+0000", 2024-04-26T04:33:02.819 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3065e-05 2024-04-26T04:33:02.819 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.819 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.819 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.819 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.201007+0000", 2024-04-26T04:33:02.819 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1333000000000001e-05 2024-04-26T04:33:02.819 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.819 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.819 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.819 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.201181+0000", 2024-04-26T04:33:02.819 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017403899999999999 2024-04-26T04:33:02.819 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.201416+0000", 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000234713 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.202587+0000", 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001171727 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.345497+0000", 2024-04-26T04:33:02.820 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.142909339 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.511984+0000", 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.16648707800000001 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.512186+0000", 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000202442 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.821 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:245 3.8s0 3:148e36d6:::smithi17829308-35:2 [read 0~573440,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.768406+0000", 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 33.043690351999999, 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30361496399999999, 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 245 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.822 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.823 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.823 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.823 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768406+0000", 2024-04-26T04:33:02.823 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.823 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.823 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.824 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.824 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768406+0000", 2024-04-26T04:33:02.824 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.824 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.824 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.824 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.824 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768408+0000", 2024-04-26T04:33:02.824 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4670000000000001e-06 2024-04-26T04:33:02.824 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.824 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.824 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.825 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768417+0000", 2024-04-26T04:33:02.825 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3230000000000004e-06 2024-04-26T04:33:02.825 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.825 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.825 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.825 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768459+0000", 2024-04-26T04:33:02.825 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1820999999999998e-05 2024-04-26T04:33:02.825 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.825 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.825 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.826 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768483+0000", 2024-04-26T04:33:02.826 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4307999999999998e-05 2024-04-26T04:33:02.826 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.826 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.826 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.826 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769025+0000", 2024-04-26T04:33:02.826 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000542086 2024-04-26T04:33:02.826 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.826 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.826 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769073+0000", 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.7707999999999998e-05 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072021+0000", 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.302948251 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:246 3.8s0 3:148e36d6:::smithi17829308-35:2 [read 0~573440,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:02.827 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.768569+0000", 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 33.043527457000003, 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30420794400000001, 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 246 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768569+0000", 2024-04-26T04:33:02.828 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768569+0000", 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768570+0000", 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2270000000000001e-06 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.829 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768578+0000", 2024-04-26T04:33:02.830 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8520000000000004e-06 2024-04-26T04:33:02.830 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.830 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.830 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.830 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768600+0000", 2024-04-26T04:33:02.830 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1948999999999999e-05 2024-04-26T04:33:02.830 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.830 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.830 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.830 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.768618+0000", 2024-04-26T04:33:02.830 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7615999999999999e-05 2024-04-26T04:33:02.830 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.830 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.831 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.831 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769307+0000", 2024-04-26T04:33:02.831 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00068885099999999998 2024-04-26T04:33:02.831 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.831 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.831 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.831 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.769389+0000", 2024-04-26T04:33:02.831 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2391000000000004e-05 2024-04-26T04:33:02.831 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.831 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.831 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.831 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072777+0000", 2024-04-26T04:33:02.831 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30338805800000002 2024-04-26T04:33:02.832 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.832 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.832 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.832 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.832 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.832 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:247 3.8s0 3:1a4ccad9:::smithi17829308-40:head [append 0~573440] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:02.832 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.788189+0000", 2024-04-26T04:33:02.832 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 33.023907512999997, 2024-04-26T04:33:02.832 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.58923427699999997, 2024-04-26T04:33:02.832 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.832 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.832 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:02.832 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.833 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.833 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.833 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 247 2024-04-26T04:33:02.833 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.833 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.833 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.833 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.833 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.788189+0000", 2024-04-26T04:33:02.834 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.834 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.834 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.834 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.834 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.788189+0000", 2024-04-26T04:33:02.834 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.834 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.834 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.834 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.834 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.788190+0000", 2024-04-26T04:33:02.834 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0270000000000001e-06 2024-04-26T04:33:02.835 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.835 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.835 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.835 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.789159+0000", 2024-04-26T04:33:02.835 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00096895399999999995 2024-04-26T04:33:02.835 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.835 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.835 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.835 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.789190+0000", 2024-04-26T04:33:02.835 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0839000000000001e-05 2024-04-26T04:33:02.835 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.835 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.836 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.836 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.789205+0000", 2024-04-26T04:33:02.836 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5139e-05 2024-04-26T04:33:02.836 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.836 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.836 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.836 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072824+0000", 2024-04-26T04:33:02.836 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.283618659 2024-04-26T04:33:02.836 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.836 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.836 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.836 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072888+0000", 2024-04-26T04:33:02.837 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.4080999999999999e-05 2024-04-26T04:33:02.837 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.837 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.837 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.837 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.073554+0000", 2024-04-26T04:33:02.837 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00066656699999999996 2024-04-26T04:33:02.837 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.837 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.838 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.838 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.077266+0000", 2024-04-26T04:33:02.838 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0037117320000000001 2024-04-26T04:33:02.838 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.838 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.838 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:02.838 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.377359+0000", 2024-04-26T04:33:02.838 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30009325399999998 2024-04-26T04:33:02.838 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.377423+0000", 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.4024999999999998e-05 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:248 3.8s0 3:1a4ccad9:::smithi17829308-40:head [append 0~376832] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.793207+0000", 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 33.018889497000004, 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.68466394600000002, 2024-04-26T04:33:02.839 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 248 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793207+0000", 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.840 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.841 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.841 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793207+0000", 2024-04-26T04:33:02.841 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.841 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.841 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.841 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.841 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793209+0000", 2024-04-26T04:33:02.841 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7269999999999999e-06 2024-04-26T04:33:02.841 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.841 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.841 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.842 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793709+0000", 2024-04-26T04:33:02.842 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000500247 2024-04-26T04:33:02.842 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.842 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.842 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.842 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793730+0000", 2024-04-26T04:33:02.842 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0931999999999999e-05 2024-04-26T04:33:02.842 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.842 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.842 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.842 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793741+0000", 2024-04-26T04:33:02.842 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1032999999999999e-05 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.074358+0000", 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.28061672700000001 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.074434+0000", 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.6691000000000001e-05 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.843 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.075231+0000", 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00079667900000000001 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.077267+0000", 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0020362380000000001 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.477751+0000", 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.400483435 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.844 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.477871+0000", 2024-04-26T04:33:02.845 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012023699999999999 2024-04-26T04:33:02.845 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.845 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.845 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.845 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.845 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.845 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:249 3.8s0 3:1a4ccad9:::smithi17829308-40:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:02.845 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.793759+0000", 2024-04-26T04:33:02.845 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 33.018337459000001, 2024-04-26T04:33:02.845 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.68433669600000002, 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 249 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793759+0000", 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.846 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793759+0000", 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793760+0000", 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.043e-06 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793765+0000", 2024-04-26T04:33:02.847 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.51e-06 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793775+0000", 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0586e-05 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893886+0000", 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100111197 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.848 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.075921+0000", 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.18203503800000001 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.076055+0000", 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013402999999999999 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.076679+0000", 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00062310000000000002 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.849 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.178214+0000", 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101535002 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478030+0000", 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.29981632200000002 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478096+0000", 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.5868e-05 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.850 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:250 3.8s0 3:1a4ccad9:::smithi17829308-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.893915+0000", 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 32.918182004999998, 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.66902760400000005, 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 250 2024-04-26T04:33:02.851 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893915+0000", 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893915+0000", 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.852 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893917+0000", 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3029999999999998e-06 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893925+0000", 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4370000000000007e-06 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893965+0000", 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9941000000000003e-05 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.853 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893976+0000", 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1049999999999999e-05 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.177679+0000", 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.28370258100000001 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.177965+0000", 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000286309 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.854 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.855 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.855 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478480+0000", 2024-04-26T04:33:02.855 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30051439600000002 2024-04-26T04:33:02.855 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.855 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.855 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.855 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478666+0000", 2024-04-26T04:33:02.855 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018635599999999999 2024-04-26T04:33:02.855 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.855 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.562942+0000", 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.084276230999999993 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:472 3.8s0 3:1d1feae2:::smithi17829308-1:head [append 0~688128] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:50.191944+0000", 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.620152143, 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.399307472, 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.856 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 472 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.191944+0000", 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.857 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.191944+0000", 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.191947+0000", 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.01e-06 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.193010+0000", 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0010629599999999999 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.858 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.193042+0000", 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1334999999999997e-05 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.193061+0000", 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9432999999999998e-05 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.285076+0000", 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.092015206000000002 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.859 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.285194+0000", 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000117422 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.286738+0000", 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001544557 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.396638+0000", 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.109899996 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.860 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.591133+0000", 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19449470299999999 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.591252+0000", 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00011885 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:473 3.8s0 3:1d1feae2:::smithi17829308-1:head [append 0~540672] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:02.861 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:50.195861+0000", 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.616235108, 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.39564706900000002, 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 473 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.195861+0000", 2024-04-26T04:33:02.862 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.195861+0000", 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.195863+0000", 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.629e-06 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.196734+0000", 2024-04-26T04:33:02.863 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00087080299999999996 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.196757+0000", 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3309e-05 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.196769+0000", 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.184e-05 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.864 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.288007+0000", 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.091238373999999997 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.288255+0000", 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024784900000000002 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.290014+0000", 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0017592230000000001 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.865 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.396649+0000", 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.106634174 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.591413+0000", 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.194764204 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.591508+0000", 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.5663999999999998e-05 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.866 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:474 3.8s0 3:1d1feae2:::smithi17829308-1:head [append 0~229376] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:50.197577+0000", 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.614519586, 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.394282619, 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 474 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.867 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.197577+0000", 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.197577+0000", 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.197579+0000", 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.903e-06 2024-04-26T04:33:02.868 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.197980+0000", 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00040159400000000002 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198001+0000", 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0842000000000001e-05 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198012+0000", 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.059e-05 2024-04-26T04:33:02.869 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.291708+0000", 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.093696404999999996 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.291839+0000", 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013111899999999999 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.292977+0000", 2024-04-26T04:33:02.870 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001137924 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.396651+0000", 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.103673705 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.591764+0000", 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19511327000000001 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.591860+0000", 2024-04-26T04:33:02.871 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.5266999999999994e-05 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:475 3.8s0 3:1d1feae2:::smithi17829308-1:head [setxattr _header (56) in=7b] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:50.198029+0000", 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.614067151, 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.39986491299999999, 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.872 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 475 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198029+0000", 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198029+0000", 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.873 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198031+0000", 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1429999999999999e-06 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198046+0000", 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5123000000000001e-05 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198056+0000", 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0023e-05 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.874 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198063+0000", 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2629999999999997e-06 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.294144+0000", 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.096080856000000006 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.294294+0000", 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014994499999999999 2024-04-26T04:33:02.875 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.295203+0000", 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00090927399999999995 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.396653+0000", 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101450493 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.597719+0000", 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201065415 2024-04-26T04:33:02.876 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.597894+0000", 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017537799999999999 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:476 3.8s0 3:1d1feae2:::smithi17829308-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:50.198078+0000", 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.614018196, 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70227189199999995, 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.877 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 476 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198078+0000", 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.878 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198078+0000", 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198079+0000", 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.2299999999999999e-07 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198082+0000", 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1700000000000001e-06 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.879 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198096+0000", 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3492999999999999e-05 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198104+0000", 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.6799999999999993e-06 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.395804+0000", 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.197700243 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.880 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.395966+0000", 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016191799999999999 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.598550+0000", 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.202584176 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.598687+0000", 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000136716 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.881 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.900350+0000", 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30166357300000002 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:623 3.8s0 3:148e36d6:::smithi17829308-35:head [append 0~622592] snapc e=[e,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.814974+0000", 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.9971227970000003, 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.724759025, 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.882 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 623 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.814974+0000", 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.883 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.814974+0000", 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.814975+0000", 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2330000000000001e-06 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.815521+0000", 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000546052 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.884 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.815539+0000", 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7765e-05 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.815549+0000", 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.022e-05 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.321677+0000", 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.50612808300000001 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.885 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.321994+0000", 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000316514 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.324954+0000", 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.002960383 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.331369+0000", 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0064146660000000003 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.886 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.539554+0000", 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20818551699999999 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.539733+0000", 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017859199999999999 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:624 3.8s0 3:148e36d6:::smithi17829308-35:head [append 0~442368] snapc e=[e,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:02.887 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.817427+0000", 2024-04-26T04:33:02.888 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.9946699240000001, 2024-04-26T04:33:02.888 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1242959029999999, 2024-04-26T04:33:02.888 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.888 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.888 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:02.888 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.888 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.888 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.888 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 624 2024-04-26T04:33:02.888 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.817427+0000", 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.817427+0000", 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.817428+0000", 2024-04-26T04:33:02.889 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8509999999999999e-06 2024-04-26T04:33:02.890 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.890 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.890 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.890 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.817869+0000", 2024-04-26T04:33:02.890 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00044101299999999997 2024-04-26T04:33:02.890 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.890 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.890 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.890 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.817888+0000", 2024-04-26T04:33:02.890 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8805999999999999e-05 2024-04-26T04:33:02.890 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.890 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.817899+0000", 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0237e-05 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.535834+0000", 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.71793524399999997 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.536014+0000", 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018016199999999999 2024-04-26T04:33:02.891 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.537405+0000", 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0013915850000000001 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.544269+0000", 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.006863231 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.941591+0000", 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.39732218800000002 2024-04-26T04:33:02.892 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.941723+0000", 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000131586 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:625 3.8s0 3:148e36d6:::smithi17829308-35:head [append 0~524288] snapc e=[e,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.820584+0000", 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.9915124879999997, 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.121569483, 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.893 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 625 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.820584+0000", 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.894 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.820584+0000", 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.820585+0000", 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4589999999999999e-06 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821101+0000", 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00051561399999999996 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.895 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.896 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821119+0000", 2024-04-26T04:33:02.896 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8054999999999999e-05 2024-04-26T04:33:02.896 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.896 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.896 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.897 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821130+0000", 2024-04-26T04:33:02.897 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1089e-05 2024-04-26T04:33:02.897 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.897 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.898 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.898 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.539869+0000", 2024-04-26T04:33:02.898 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.71873915499999996 2024-04-26T04:33:02.898 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.898 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.898 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.898 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.540052+0000", 2024-04-26T04:33:02.898 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018266199999999999 2024-04-26T04:33:02.898 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.898 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.898 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.898 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.541436+0000", 2024-04-26T04:33:02.898 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001384363 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.545635+0000", 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0041985219999999997 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.941934+0000", 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.39629892900000002 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.942154+0000", 2024-04-26T04:33:02.899 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021963500000000001 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:626 3.8s0 3:148e36d6:::smithi17829308-35:head [append 0~196608] snapc e=[e,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.821693+0000", 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.9904035090000001, 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1208058139999999, 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.900 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 626 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821693+0000", 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821693+0000", 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.901 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821694+0000", 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0210000000000001e-06 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821910+0000", 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021605499999999999 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821925+0000", 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4931e-05 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.902 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821933+0000", 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1890000000000007e-06 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.542462+0000", 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.72052851399999995 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.542611+0000", 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014969199999999999 2024-04-26T04:33:02.903 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.543633+0000", 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001021599 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.545642+0000", 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0020087630000000002 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.942386+0000", 2024-04-26T04:33:02.904 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.396744659 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.942499+0000", 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000112391 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:627 3.8s0 3:148e36d6:::smithi17829308-35:head [setxattr _header (57) in=7b] snapc e=[e,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.821959+0000", 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.9901372850000003, 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1209552119999999, 2024-04-26T04:33:02.905 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 627 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821959+0000", 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.906 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821959+0000", 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821960+0000", 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2099999999999996e-07 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821964+0000", 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8759999999999998e-06 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.907 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821974+0000", 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.8169999999999994e-06 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821982+0000", 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8550000000000002e-06 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.544392+0000", 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.72241027499999999 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.908 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.544530+0000", 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000137832 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.545257+0000", 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000727351 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.547660+0000", 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0024034809999999998 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.909 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.942747+0000", 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.39508694900000002 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.942914+0000", 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016705499999999999 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.910 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:628 3.8s0 3:148e36d6:::smithi17829308-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.821991+0000", 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.9901057919999996, 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3245125230000001, 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 628 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.911 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821991+0000", 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821991+0000", 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821991+0000", 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.1900000000000003e-07 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.912 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821994+0000", 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3429999999999998e-06 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.822002+0000", 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6750000000000008e-06 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922073+0000", 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100070414 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.913 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636093+0000", 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.71402036099999999 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636365+0000", 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00027212599999999998 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.044197+0000", 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.40783160699999998 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.914 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.044341+0000", 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000144463 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.146503+0000", 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10216201499999999 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.0s4 60/30 ECSubWrite(tid=31, reqid=client.4670.0:605, at_version=60'14, trim_to=0'0, roll_forward_to=59'11))", 2024-04-26T04:33:02.915 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.922386+0000", 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.8897105700000001, 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.40454453299999998, 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922386+0000", 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922386+0000", 2024-04-26T04:33:02.916 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922386+0000", 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0899999999999997e-07 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922407+0000", 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1024999999999999e-05 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922415+0000", 2024-04-26T04:33:02.917 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.96e-06 2024-04-26T04:33:02.918 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.918 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.918 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.918 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922420+0000", 2024-04-26T04:33:02.918 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.5159999999999996e-06 2024-04-26T04:33:02.918 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.918 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.918 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.918 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.114656+0000", 2024-04-26T04:33:02.918 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.192236623 2024-04-26T04:33:02.918 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.114686+0000", 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0105999999999999e-05 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.326873+0000", 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.212186545 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.326930+0000", 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.7448999999999998e-05 2024-04-26T04:33:02.919 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.920 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.920 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.920 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.920 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.920 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.0s4 60/30 ECSubWrite(tid=32, reqid=client.4670.0:606, at_version=60'15, trim_to=0'0, roll_forward_to=59'11))", 2024-04-26T04:33:02.920 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:56.012226+0000", 2024-04-26T04:33:02.920 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.7998701439999998, 2024-04-26T04:33:02.920 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.314742413, 2024-04-26T04:33:02.920 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:02.920 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.012226+0000", 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.012226+0000", 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.921 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.012228+0000", 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7829999999999999e-06 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.012263+0000", 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.5219000000000003e-05 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.012295+0000", 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1418999999999999e-05 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.922 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:02.923 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.012313+0000", 2024-04-26T04:33:02.923 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8542999999999998e-05 2024-04-26T04:33:02.923 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.923 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.923 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:02.923 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.115720+0000", 2024-04-26T04:33:02.923 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10340703399999999 2024-04-26T04:33:02.923 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.923 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.925 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:02.925 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.115738+0000", 2024-04-26T04:33:02.925 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7212999999999999e-05 2024-04-26T04:33:02.925 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.925 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.925 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:02.925 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.326937+0000", 2024-04-26T04:33:02.925 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.211199469 2024-04-26T04:33:02.925 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:02.926 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:02.926 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:02.926 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.326969+0000", 2024-04-26T04:33:02.926 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1733000000000003e-05 2024-04-26T04:33:02.926 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.926 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.926 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.926 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:02.926 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:02.926 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:02.928 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T04:33:02.928 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:02.926+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:03.028 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:03.024+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:03.054 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:03.054 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:33:03.054 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 0 2024-04-26T04:33:03.054 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:03.068 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T04:33:03.130 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:03.129+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:03.187 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:03.187 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:33:03.187 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:33:03.187 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:33:03.187 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:03.202 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T04:33:03.224 INFO:tasks.rados.rados.0.smithi178.stdout:295: done (9 left) 2024-04-26T04:33:03.224 INFO:tasks.rados.rados.0.smithi178.stdout:296: copy_from oid 32 from oid 9 current snap is 16 2024-04-26T04:33:03.224 INFO:tasks.rados.rados.0.smithi178.stdout:284: finishing write tid 3 to smithi17829308-49 2024-04-26T04:33:03.225 INFO:tasks.rados.rados.0.smithi178.stdout:284: finishing write tid 4 to smithi17829308-49 2024-04-26T04:33:03.225 INFO:tasks.rados.rados.0.smithi178.stdout:284: finishing write tid 5 to smithi17829308-49 2024-04-26T04:33:03.225 INFO:tasks.rados.rados.0.smithi178.stdout:280: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:33:03.247 INFO:tasks.rados.rados.0.smithi178.stdout:284: finishing write tid 6 to smithi17829308-49 2024-04-26T04:33:03.247 INFO:tasks.rados.rados.0.smithi178.stdout:284: oid 49 updating version 0 to 40 2024-04-26T04:33:03.247 INFO:tasks.rados.rados.0.smithi178.stdout:284: oid 49 updating version 40 to 43 2024-04-26T04:33:03.247 INFO:tasks.rados.rados.0.smithi178.stdout:284: oid 49 version 43 is already newer than 41 2024-04-26T04:33:03.247 INFO:tasks.rados.rados.0.smithi178.stdout:284: oid 49 version 43 is already newer than 42 2024-04-26T04:33:03.248 INFO:tasks.rados.rados.0.smithi178.stdout:284: oid 49 version 43 is already newer than 39 2024-04-26T04:33:03.248 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 49 v 43 (ObjNum 117 snap 16 seq_num 117) dirty exists 2024-04-26T04:33:03.248 INFO:tasks.rados.rados.0.smithi178.stdout:284: left oid 49 (ObjNum 117 snap 16 seq_num 117) 2024-04-26T04:33:03.248 INFO:tasks.rados.rados.0.smithi178.stdout:291: finishing write tid 1 to smithi17829308-4 2024-04-26T04:33:03.248 INFO:tasks.rados.rados.0.smithi178.stdout:288: expect (ObjNum 31 snap 0 seq_num 31) 2024-04-26T04:33:03.248 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:03.249+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:03.266 INFO:tasks.rados.rados.0.smithi178.stdout:291: finishing write tid 2 to smithi17829308-4 2024-04-26T04:33:03.266 INFO:tasks.rados.rados.0.smithi178.stdout:294: finishing write tid 1 to smithi17829308-33 2024-04-26T04:33:03.266 INFO:tasks.rados.rados.0.smithi178.stdout:294: finishing write tid 2 to smithi17829308-33 2024-04-26T04:33:03.266 INFO:tasks.rados.rados.0.smithi178.stdout:294: finishing write tid 3 to smithi17829308-33 2024-04-26T04:33:03.266 INFO:tasks.rados.rados.0.smithi178.stdout:294: finishing write tid 4 to smithi17829308-33 2024-04-26T04:33:03.266 INFO:tasks.rados.rados.0.smithi178.stdout:294: finishing write tid 5 to smithi17829308-33 2024-04-26T04:33:03.266 INFO:tasks.rados.rados.0.smithi178.stdout:293: finishing copy_from racing read to smithi17829308-10 2024-04-26T04:33:03.266 INFO:tasks.rados.rados.0.smithi178.stdout:287: finishing copy_from racing read to smithi17829308-44 2024-04-26T04:33:03.266 INFO:tasks.rados.rados.0.smithi178.stdout:291: finishing write tid 3 to smithi17829308-4 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:291: finishing write tid 4 to smithi17829308-4 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:291: finishing write tid 5 to smithi17829308-4 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:294: finishing write tid 6 to smithi17829308-33 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:294: oid 33 updating version 0 to 41 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:294: oid 33 updating version 41 to 45 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:294: oid 33 version 45 is already newer than 43 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:294: oid 33 version 45 is already newer than 42 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:294: oid 33 version 45 is already newer than 44 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 33 v 45 (ObjNum 123 snap 16 seq_num 123) dirty exists 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:294: left oid 33 (ObjNum 123 snap 16 seq_num 123) 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:292: finishing write tid 1 to smithi17829308-20 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:292: finishing write tid 2 to smithi17829308-20 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:291: finishing write tid 6 to smithi17829308-4 2024-04-26T04:33:03.267 INFO:tasks.rados.rados.0.smithi178.stdout:291: oid 4 updating version 0 to 65 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:291: oid 4 version 65 is already newer than 64 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:291: oid 4 updating version 65 to 66 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:291: oid 4 version 66 is already newer than 63 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:291: oid 4 updating version 66 to 67 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 4 v 67 (ObjNum 121 snap 16 seq_num 121) dirty exists 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:291: left oid 4 (ObjNum 121 snap 16 seq_num 121) 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:287: finishing copy_from to smithi17829308-44 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 44 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:293: finishing copy_from to smithi17829308-10 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 10 v 2 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:292: finishing write tid 3 to smithi17829308-20 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:292: finishing write tid 4 to smithi17829308-20 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:292: finishing write tid 5 to smithi17829308-20 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:292: finishing write tid 6 to smithi17829308-20 2024-04-26T04:33:03.268 INFO:tasks.rados.rados.0.smithi178.stdout:280: done (9 left) 2024-04-26T04:33:03.269 INFO:tasks.rados.rados.0.smithi178.stdout:284: done (8 left) 2024-04-26T04:33:03.269 INFO:tasks.rados.rados.0.smithi178.stdout:287: done (7 left) 2024-04-26T04:33:03.269 INFO:tasks.rados.rados.0.smithi178.stdout:288: done (6 left) 2024-04-26T04:33:03.269 INFO:tasks.rados.rados.0.smithi178.stdout:291: done (5 left) 2024-04-26T04:33:03.269 INFO:tasks.rados.rados.0.smithi178.stdout:293: done (4 left) 2024-04-26T04:33:03.269 INFO:tasks.rados.rados.0.smithi178.stdout:294: done (3 left) 2024-04-26T04:33:03.269 INFO:tasks.rados.rados.0.smithi178.stdout:297: snap_remove snap 12 2024-04-26T04:33:03.329 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:03.329 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:33:03.329 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:33:03.329 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:33:03.329 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.329 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 32/30 ECSubWrite(tid=1, reqid=client.4670.0:11, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:03.329 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.209171+0000", 2024-04-26T04:33:03.329 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 38.119417257000002, 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19807992999999999, 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209171+0000", 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209171+0000", 2024-04-26T04:33:03.330 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209173+0000", 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1440000000000001e-06 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209220+0000", 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.7030000000000002e-05 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.331 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209250+0000", 2024-04-26T04:33:03.333 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0281999999999999e-05 2024-04-26T04:33:03.333 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.333 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.333 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.333 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.209281+0000", 2024-04-26T04:33:03.333 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0895000000000002e-05 2024-04-26T04:33:03.333 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.333 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.334 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.334 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.405953+0000", 2024-04-26T04:33:03.334 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19667172599999999 2024-04-26T04:33:03.334 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.334 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.334 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.334 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.405979+0000", 2024-04-26T04:33:03.334 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6452e-05 2024-04-26T04:33:03.334 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.334 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.334 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.334 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.407083+0000", 2024-04-26T04:33:03.334 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0011043139999999999 2024-04-26T04:33:03.335 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.335 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.335 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.335 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.407250+0000", 2024-04-26T04:33:03.335 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016708700000000001 2024-04-26T04:33:03.335 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.335 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.335 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.335 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.335 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.335 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s2 32/30 ECSubWrite(tid=1, reqid=client.4670.0:5, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:03.335 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.312729+0000", 2024-04-26T04:33:03.336 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 38.015858956999999, 2024-04-26T04:33:03.336 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20126603100000001, 2024-04-26T04:33:03.336 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.336 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.336 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.336 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.336 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.336 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.336 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312729+0000", 2024-04-26T04:33:03.336 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.336 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.336 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.336 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.337 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312729+0000", 2024-04-26T04:33:03.337 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.337 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.337 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.337 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.337 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312731+0000", 2024-04-26T04:33:03.337 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9130000000000001e-06 2024-04-26T04:33:03.337 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.337 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.337 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.337 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312766+0000", 2024-04-26T04:33:03.337 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4768000000000003e-05 2024-04-26T04:33:03.337 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312797+0000", 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1201000000000002e-05 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312814+0000", 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7073000000000001e-05 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.413078+0000", 2024-04-26T04:33:03.338 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10026389400000001 2024-04-26T04:33:03.339 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.339 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.339 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.339 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.413092+0000", 2024-04-26T04:33:03.339 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4188999999999999e-05 2024-04-26T04:33:03.339 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.339 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.339 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.339 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.513794+0000", 2024-04-26T04:33:03.339 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100702298 2024-04-26T04:33:03.339 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.339 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.513995+0000", 2024-04-26T04:33:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000200695 2024-04-26T04:33:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.7s2 35/30 ECSubRead(tid=58, to_read={3:e1054bf1:::smithi17829308-37:head=0,151552,0}, subchunks={3:e1054bf1:::smithi17829308-37:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:33:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.954839+0000", 2024-04-26T04:33:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 34.373749216999997, 2024-04-26T04:33:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20146535800000001, 2024-04-26T04:33:03.340 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954839+0000", 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954839+0000", 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.341 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954840+0000", 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.826e-06 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954857+0000", 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6333000000000001e-05 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954890+0000", 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.3594999999999999e-05 2024-04-26T04:33:03.342 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954912+0000", 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1325e-05 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.155338+0000", 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20042677 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.156304+0000", 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00096550900000000003 2024-04-26T04:33:03.343 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs6 35/30 ECSubWrite(tid=59, reqid=client.4670.0:176, at_version=0'0, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.055093+0000", 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 34.273495230000002, 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201719765, 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.344 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055093+0000", 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055093+0000", 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055095+0000", 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.227e-06 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.345 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055119+0000", 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4329e-05 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055151+0000", 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1507999999999997e-05 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.155266+0000", 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100114996 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.346 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.255601+0000", 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100334886 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.255617+0000", 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6446000000000001e-05 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.256718+0000", 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001101524 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.347 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.256812+0000", 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3849000000000002e-05 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es1 35/30 ECSubWrite(tid=190, reqid=client.4670.0:235, at_version=0'0, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.421980+0000", 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 33.906608065, 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20113165399999999, 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.348 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421980+0000", 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421980+0000", 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.349 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421982+0000", 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2639999999999998e-06 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.422038+0000", 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.6273000000000001e-05 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.422071+0000", 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2437999999999997e-05 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.350 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.522157+0000", 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100086417 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.622323+0000", 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100166121 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.622332+0000", 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.1309999999999995e-06 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.351 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.623023+0000", 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000690491 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.623111+0000", 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.8518999999999997e-05 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s2 41/30 ECSubWrite(tid=115, reqid=client.4670.0:355, at_version=41'36, trim_to=0'0, roll_forward_to=40'34))", 2024-04-26T04:33:03.352 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.444369+0000", 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 27.884219151, 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20274193700000001, 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444369+0000", 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.353 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444369+0000", 2024-04-26T04:33:03.354 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.354 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.354 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.354 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.354 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444371+0000", 2024-04-26T04:33:03.354 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9099999999999999e-06 2024-04-26T04:33:03.354 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.354 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.354 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.354 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444630+0000", 2024-04-26T04:33:03.354 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00025897499999999999 2024-04-26T04:33:03.354 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.354 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444655+0000", 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5819000000000001e-05 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444670+0000", 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4779e-05 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.644608+0000", 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19993833 2024-04-26T04:33:03.355 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.644623+0000", 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4806e-05 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.647032+0000", 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0024082880000000002 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.647111+0000", 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.9029999999999994e-05 2024-04-26T04:33:03.356 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.357 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.357 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.357 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.357 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.357 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s2 41/30 ECSubWrite(tid=116, reqid=client.4670.0:356, at_version=41'37, trim_to=0'0, roll_forward_to=40'34))", 2024-04-26T04:33:03.357 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.447328+0000", 2024-04-26T04:33:03.357 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 27.881260138000002, 2024-04-26T04:33:03.357 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.200159164, 2024-04-26T04:33:03.357 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.357 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.357 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.357 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.357 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.447328+0000", 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.447328+0000", 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.447329+0000", 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6500000000000001e-06 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.358 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.359 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.359 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.447476+0000", 2024-04-26T04:33:03.359 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014682700000000001 2024-04-26T04:33:03.359 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.359 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.359 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.359 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.447506+0000", 2024-04-26T04:33:03.359 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9694999999999999e-05 2024-04-26T04:33:03.359 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.359 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.359 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.359 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.447521+0000", 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5383000000000002e-05 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.645443+0000", 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19792164700000001 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.645469+0000", 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6602999999999999e-05 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.360 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.647413+0000", 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001943492 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.647487+0000", 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3867000000000003e-05 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.7s2 45/30 ECSubWrite(tid=137, reqid=client.4670.0:371, at_version=45'18, trim_to=0'0, roll_forward_to=44'15))", 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.608809+0000", 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 23.719778900000001, 2024-04-26T04:33:03.361 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20229014100000001, 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608809+0000", 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608809+0000", 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.362 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608811+0000", 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2730000000000001e-06 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609164+0000", 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00035305999999999998 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609195+0000", 2024-04-26T04:33:03.363 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0818000000000002e-05 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609214+0000", 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8819000000000001e-05 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.709463+0000", 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10024947300000001 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.709490+0000", 2024-04-26T04:33:03.364 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6312000000000002e-05 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.811034+0000", 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10154439899999999 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.811099+0000", 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.4987000000000006e-05 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.365 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.366 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.7s2 45/30 ECSubWrite(tid=138, reqid=client.4670.0:372, at_version=45'19, trim_to=0'0, roll_forward_to=44'15))", 2024-04-26T04:33:03.366 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.609251+0000", 2024-04-26T04:33:03.366 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 23.719336405, 2024-04-26T04:33:03.366 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20191711300000001, 2024-04-26T04:33:03.366 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.366 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.366 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.366 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.366 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.366 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.366 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609251+0000", 2024-04-26T04:33:03.366 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.366 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609251+0000", 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609252+0000", 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2999999999999999e-07 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609276+0000", 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3703e-05 2024-04-26T04:33:03.367 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609292+0000", 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6535999999999999e-05 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609303+0000", 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0539e-05 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.710393+0000", 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101089625 2024-04-26T04:33:03.368 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.710408+0000", 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5079000000000001e-05 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.811107+0000", 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10069903099999999 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.811169+0000", 2024-04-26T04:33:03.369 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.177e-05 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 45/30 ECSubWrite(tid=35, reqid=client.4670.0:377, at_version=0'0, trim_to=0'0, roll_forward_to=45'23))", 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:40.134471+0000", 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 23.194116606000001, 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20268414700000001, 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.370 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134471+0000", 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134471+0000", 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134473+0000", 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2000000000000001e-06 2024-04-26T04:33:03.371 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134501+0000", 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7441999999999999e-05 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134540+0000", 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9181999999999999e-05 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234639+0000", 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100099332 2024-04-26T04:33:03.372 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234798+0000", 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015831999999999999 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234822+0000", 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4766e-05 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.337076+0000", 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102253652 2024-04-26T04:33:03.373 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.337155+0000", 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.9252999999999995e-05 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 45/30 ECSubWrite(tid=36, reqid=client.4670.0:379, at_version=45'25, trim_to=0'0, roll_forward_to=45'23))", 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:40.234694+0000", 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 23.093893675, 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.2029495, 2024-04-26T04:33:03.374 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234694+0000", 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234694+0000", 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.375 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.376 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.376 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234695+0000", 2024-04-26T04:33:03.376 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.356e-06 2024-04-26T04:33:03.376 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.376 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.376 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.376 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235050+0000", 2024-04-26T04:33:03.376 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00035439899999999998 2024-04-26T04:33:03.376 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.376 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.376 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.376 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235076+0000", 2024-04-26T04:33:03.376 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5619e-05 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235096+0000", 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0046000000000001e-05 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.335458+0000", 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10036199799999999 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.377 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.335472+0000", 2024-04-26T04:33:03.378 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.472e-05 2024-04-26T04:33:03.378 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.378 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.378 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.378 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.437566+0000", 2024-04-26T04:33:03.378 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102093589 2024-04-26T04:33:03.378 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.378 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.378 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.378 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.437644+0000", 2024-04-26T04:33:03.378 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7773000000000002e-05 2024-04-26T04:33:03.378 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.378 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.4s1 45/30 ECSubRead(tid=37, to_read={3:26eb43ca:::smithi17829308-16:head=0,4096,0}, subchunks={3:26eb43ca:::smithi17829308-16:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:40.235130+0000", 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 23.093458052999999, 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20238550699999999, 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235130+0000", 2024-04-26T04:33:03.379 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235130+0000", 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235131+0000", 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1739999999999999e-06 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235146+0000", 2024-04-26T04:33:03.380 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5175e-05 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235172+0000", 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5406e-05 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235195+0000", 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3266000000000001e-05 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.437356+0000", 2024-04-26T04:33:03.381 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20216114600000001 2024-04-26T04:33:03.382 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.382 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.382 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.382 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.437515+0000", 2024-04-26T04:33:03.382 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015934 2024-04-26T04:33:03.382 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.382 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.382 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.382 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.382 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.382 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 47/30 ECSubWrite(tid=42, reqid=osd.8.0:42, at_version=47'31, trim_to=0'0, roll_forward_to=46'29))", 2024-04-26T04:33:03.382 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:41.248219+0000", 2024-04-26T04:33:03.382 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.080368932999999, 2024-04-26T04:33:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20129562100000001, 2024-04-26T04:33:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248219+0000", 2024-04-26T04:33:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.383 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248219+0000", 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248220+0000", 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5400000000000001e-06 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248271+0000", 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.0377999999999998e-05 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.384 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.385 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248305+0000", 2024-04-26T04:33:03.385 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4598999999999999e-05 2024-04-26T04:33:03.385 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.385 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.385 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.385 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348396+0000", 2024-04-26T04:33:03.385 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10009045699999999 2024-04-26T04:33:03.385 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.385 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.385 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.385 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348535+0000", 2024-04-26T04:33:03.385 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000138975 2024-04-26T04:33:03.385 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.386 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.386 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.386 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348560+0000", 2024-04-26T04:33:03.386 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5333e-05 2024-04-26T04:33:03.386 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.386 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.386 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.386 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.449434+0000", 2024-04-26T04:33:03.386 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100873729 2024-04-26T04:33:03.386 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.386 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.386 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.386 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.449514+0000", 2024-04-26T04:33:03.387 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0610000000000002e-05 2024-04-26T04:33:03.387 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.387 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.387 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.387 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.387 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.387 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.7s2 57/30 ECSubWrite(tid=185, reqid=osd.4.0:185, at_version=57'40, trim_to=0'0, roll_forward_to=54'38))", 2024-04-26T04:33:03.387 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:51.412798+0000", 2024-04-26T04:33:03.387 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 11.915789415999999, 2024-04-26T04:33:03.387 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201917122, 2024-04-26T04:33:03.387 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.387 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.387 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.412798+0000", 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.412798+0000", 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.412799+0000", 2024-04-26T04:33:03.388 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.3000000000000001e-07 2024-04-26T04:33:03.389 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.389 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.389 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.389 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.412836+0000", 2024-04-26T04:33:03.389 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.6808999999999999e-05 2024-04-26T04:33:03.389 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.389 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.389 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.389 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.412865+0000", 2024-04-26T04:33:03.389 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9016000000000001e-05 2024-04-26T04:33:03.389 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.389 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.389 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.512998+0000", 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100133465 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.613254+0000", 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10025622300000001 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.613269+0000", 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4946000000000001e-05 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.390 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.391 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.614645+0000", 2024-04-26T04:33:03.391 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0013753420000000001 2024-04-26T04:33:03.391 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.391 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.391 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.391 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.614716+0000", 2024-04-26T04:33:03.391 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0790999999999994e-05 2024-04-26T04:33:03.391 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.391 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.391 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.391 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.391 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.391 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.es1 61/30 ECSubRead(tid=219, to_read={3:7dd262f9:::smithi17829308-15:head=0,4096,0}, subchunks={3:7dd262f9:::smithi17829308-15:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:56.387070+0000", 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 6.9415178439999998, 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.199309295, 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.387070+0000", 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.392 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.387070+0000", 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.387071+0000", 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5269999999999999e-06 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.387089+0000", 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7932e-05 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.393 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.394 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.394 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.387120+0000", 2024-04-26T04:33:03.394 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0116000000000001e-05 2024-04-26T04:33:03.394 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.394 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.394 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.394 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.387146+0000", 2024-04-26T04:33:03.394 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6092999999999999e-05 2024-04-26T04:33:03.394 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.394 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.394 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.394 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.586113+0000", 2024-04-26T04:33:03.394 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19896773500000001 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.586379+0000", 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026589199999999999 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.as3 62/30 ECSubWrite(tid=48, reqid=client.4670.0:645, at_version=62'36, trim_to=0'0, roll_forward_to=59'35))", 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:57.510030+0000", 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.8185581700000002, 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20227371499999999, 2024-04-26T04:33:03.395 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.510030+0000", 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.510030+0000", 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.396 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.510032+0000", 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2400000000000002e-06 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.510418+0000", 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000386499 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.510449+0000", 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0335999999999999e-05 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.397 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.510466+0000", 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7385999999999999e-05 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.710808+0000", 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20034220799999999 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.710824+0000", 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5285999999999999e-05 2024-04-26T04:33:03.398 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.712235+0000", 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0014111480000000001 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.712303+0000", 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.8612e-05 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.399 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.1s2 66/30 ECSubRead(tid=260, to_read={3:8f617591:::smithi17829308-17:head=0,692224,0}, subchunks={3:8f617591:::smithi17829308-17:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:33:03.400 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:01.785908+0000", 2024-04-26T04:33:03.400 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.5426799579999999, 2024-04-26T04:33:03.400 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20040781599999999, 2024-04-26T04:33:03.400 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.400 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.400 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.400 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.400 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.400 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.400 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.785908+0000", 2024-04-26T04:33:03.400 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.400 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.400 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.785908+0000", 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.785909+0000", 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7000000000000004e-07 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.785919+0000", 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0467e-05 2024-04-26T04:33:03.401 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.402 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.402 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.402 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.785979+0000", 2024-04-26T04:33:03.402 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.0191e-05 2024-04-26T04:33:03.402 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.402 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.402 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.402 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.786001+0000", 2024-04-26T04:33:03.402 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1537000000000001e-05 2024-04-26T04:33:03.402 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.402 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.402 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.403 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.986138+0000", 2024-04-26T04:33:03.403 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20013678100000001 2024-04-26T04:33:03.403 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.403 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.403 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.403 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:01.986316+0000", 2024-04-26T04:33:03.403 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017807000000000001 2024-04-26T04:33:03.403 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.403 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.403 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.404 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.404 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.404 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.6s6 67/30 ECSubWrite(tid=59, reqid=client.4670.0:749, at_version=67'41, trim_to=0'0, roll_forward_to=65'39))", 2024-04-26T04:33:03.404 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:02.406335+0000", 2024-04-26T04:33:03.404 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.92225239000000003, 2024-04-26T04:33:03.404 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19971460199999999, 2024-04-26T04:33:03.404 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.404 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.404 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.404 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.405 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.405 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.405 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.406335+0000", 2024-04-26T04:33:03.405 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.405 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.405 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.405 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.405 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.406335+0000", 2024-04-26T04:33:03.405 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.405 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.406 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.406 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.406 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.406336+0000", 2024-04-26T04:33:03.406 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9200000000000002e-07 2024-04-26T04:33:03.406 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.406 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.406 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.406 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.406939+0000", 2024-04-26T04:33:03.406 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00060333800000000003 2024-04-26T04:33:03.406 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.406 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.406 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.406977+0000", 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.7422999999999997e-05 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.407007+0000", 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0429999999999998e-05 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.502988+0000", 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.095980987000000004 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.407 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.408 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.502999+0000", 2024-04-26T04:33:03.408 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0701e-05 2024-04-26T04:33:03.408 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.408 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.408 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.408 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.605994+0000", 2024-04-26T04:33:03.408 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10299565300000001 2024-04-26T04:33:03.408 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.408 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.408 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.408 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.606050+0000", 2024-04-26T04:33:03.408 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.5677999999999998e-05 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.6s6 67/30 ECSubWrite(tid=60, reqid=client.4670.0:750, at_version=67'42, trim_to=0'0, roll_forward_to=65'39))", 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:02.411491+0000", 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.91709725900000005, 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19459063400000001, 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.409 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.410 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.410 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.411491+0000", 2024-04-26T04:33:03.410 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.410 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.410 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.410 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.410 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.411491+0000", 2024-04-26T04:33:03.410 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.410 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.410 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.410 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.410 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.411492+0000", 2024-04-26T04:33:03.410 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1739999999999999e-06 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.411616+0000", 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000124521 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.411636+0000", 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0237e-05 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.411 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.411650+0000", 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3193999999999999e-05 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.604270+0000", 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.192620185 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.604279+0000", 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.8480000000000007e-06 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.412 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.413 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.606065+0000", 2024-04-26T04:33:03.413 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0017864370000000001 2024-04-26T04:33:03.413 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.413 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.413 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.413 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.606081+0000", 2024-04-26T04:33:03.413 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6038000000000001e-05 2024-04-26T04:33:03.413 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.413 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.413 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.413 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.413 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.413 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.6s6 67/30 ECSubWrite(tid=61, reqid=client.4670.0:751, at_version=67'43, trim_to=0'0, roll_forward_to=65'39))", 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:02.414765+0000", 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.91382291999999998, 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19150313899999999, 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.414765+0000", 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.414 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.414765+0000", 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.414766+0000", 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.049e-06 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.414994+0000", 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00022763699999999999 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.415 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.416 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.416 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.415019+0000", 2024-04-26T04:33:03.416 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5160999999999998e-05 2024-04-26T04:33:03.416 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.416 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.416 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.416 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.415034+0000", 2024-04-26T04:33:03.416 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4953999999999999e-05 2024-04-26T04:33:03.416 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.416 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.416 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.416 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.604764+0000", 2024-04-26T04:33:03.416 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.189729922 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.604774+0000", 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0848e-05 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.606245+0000", 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0014704189999999999 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.417 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.606268+0000", 2024-04-26T04:33:03.418 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3149000000000001e-05 2024-04-26T04:33:03.418 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.418 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.418 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.418 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.418 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.418 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:03.420 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T04:33:03.420 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:03.417+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:03.521 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:03.521+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:03.590 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:03.590 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:33:03.590 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.590 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 68/30 ECSubWriteReply(tid=243, last_complete=68'50, committed=1, applied=1))", 2024-04-26T04:33:03.590 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:03.443577+0000", 2024-04-26T04:33:03.590 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.14733956400000001, 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.147380237, 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.443577+0000", 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.443577+0000", 2024-04-26T04:33:03.591 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.443578+0000", 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.26e-06 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.443586+0000", 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7530000000000001e-06 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.592 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.443609+0000", 2024-04-26T04:33:03.593 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3566000000000001e-05 2024-04-26T04:33:03.593 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.593 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.593 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.593 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.443622+0000", 2024-04-26T04:33:03.593 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2830000000000001e-05 2024-04-26T04:33:03.593 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.593 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.593 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.593 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.593 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.593 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 68/30 ECSubWriteReply(tid=243, last_complete=68'50, committed=1, applied=1))", 2024-04-26T04:33:03.593 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:03.543186+0000", 2024-04-26T04:33:03.594 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.047730582000000001, 2024-04-26T04:33:03.594 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.047857368999999997, 2024-04-26T04:33:03.594 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.594 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.594 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:03.594 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.594 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.594 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.594 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.543186+0000", 2024-04-26T04:33:03.594 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.594 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.543186+0000", 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.543187+0000", 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.034e-06 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.543192+0000", 2024-04-26T04:33:03.595 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.3260000000000003e-06 2024-04-26T04:33:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.543205+0000", 2024-04-26T04:33:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2774e-05 2024-04-26T04:33:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.543213+0000", 2024-04-26T04:33:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.6109999999999996e-06 2024-04-26T04:33:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.596 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.597 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.597 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-26T04:33:03.597 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 2 2024-04-26T04:33:03.597 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:03.605 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T04:33:03.649 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:03.649+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:03.722 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:03.720+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:03.775 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:03.776 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:33:03.776 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:33:03.776 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:33:03.776 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:03.790 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T04:33:03.837 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:03.837+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:03.919 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:03.919 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:33:03.919 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:33:03.919 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:33:03.919 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.919 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-26T04:33:03.919 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:33:03.919 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 54.444798931999998, 2024-04-26T04:33:03.920 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.261775152, 2024-04-26T04:33:03.920 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.920 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.920 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:03.920 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.920 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:33:03.920 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:33:03.920 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 1 2024-04-26T04:33:03.920 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.920 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.920 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.920 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.920 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471107+0000", 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4540000000000001e-06 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.921 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471120+0000", 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3217999999999999e-05 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471147+0000", 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6461e-05 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471188+0000", 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1539000000000003e-05 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.922 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.472842+0000", 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016540700000000001 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for peered", 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.472873+0000", 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0300000000000001e-05 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.729650+0000", 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2567770170000001 2024-04-26T04:33:03.923 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.729957+0000", 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00030700500000000001 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.730324+0000", 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00036723099999999998 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732599+0000", 2024-04-26T04:33:03.924 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0022751260000000001 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732706+0000", 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010741 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732804+0000", 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7348000000000004e-05 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.925 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732880+0000", 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5972999999999999e-05 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708719+0000", 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 47.207183950000001, 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60283476700000005, 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:03.926 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.927 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:33:03.927 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:33:03.927 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 115 2024-04-26T04:33:03.927 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.927 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.927 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.927 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.927 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708719+0000", 2024-04-26T04:33:03.927 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.927 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.927 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.927 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.927 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708719+0000", 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708720+0000", 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6700000000000003e-07 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708854+0000", 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013397699999999999 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.928 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708866+0000", 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1875000000000001e-05 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708874+0000", 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0600000000000008e-06 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.818243+0000", 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10936897900000001 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.929 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.930 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.818373+0000", 2024-04-26T04:33:03.930 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012997700000000001 2024-04-26T04:33:03.930 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.930 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.930 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:33:03.930 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.818585+0000", 2024-04-26T04:33:03.930 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021234699999999999 2024-04-26T04:33:03.930 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.930 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.930 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:33:03.930 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820901+0000", 2024-04-26T04:33:03.930 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0023151669999999999 2024-04-26T04:33:03.930 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311251+0000", 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.49035071699999999 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311421+0000", 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016933399999999999 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311554+0000", 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013366699999999999 2024-04-26T04:33:03.931 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708888+0000", 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 47.207015206000001, 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60310785300000003, 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:33:03.932 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 116 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708888+0000", 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708888+0000", 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.933 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.934 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708889+0000", 2024-04-26T04:33:03.934 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8700000000000005e-07 2024-04-26T04:33:03.934 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.934 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.934 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.934 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708894+0000", 2024-04-26T04:33:03.934 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.446e-06 2024-04-26T04:33:03.934 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.934 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.934 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.934 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708904+0000", 2024-04-26T04:33:03.934 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7059999999999999e-06 2024-04-26T04:33:03.934 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708912+0000", 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3730000000000004e-06 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.819534+0000", 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.110621994 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.819754+0000", 2024-04-26T04:33:03.935 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00022034999999999999 2024-04-26T04:33:03.936 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.936 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.936 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:33:03.936 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820112+0000", 2024-04-26T04:33:03.936 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00035776299999999998 2024-04-26T04:33:03.936 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.936 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.936 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:33:03.936 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821816+0000", 2024-04-26T04:33:03.936 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001704346 2024-04-26T04:33:03.936 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.936 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.936 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311719+0000", 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.48990303600000001 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311876+0000", 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015653800000000001 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.311996+0000", 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012051400000000001 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.937 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.938 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.938 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.938 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.938 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:33:03.938 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:33:03.938 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 47.206981001000003, 2024-04-26T04:33:03.938 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60751630199999995, 2024-04-26T04:33:03.938 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.938 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.938 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:03.938 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.938 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:33:03.938 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 117 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.939 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708923+0000", 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3300000000000002e-07 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708929+0000", 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.9959999999999999e-06 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708941+0000", 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2085e-05 2024-04-26T04:33:03.940 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708948+0000", 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.8170000000000001e-06 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820967+0000", 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.112019008 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821107+0000", 2024-04-26T04:33:03.941 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000139825 2024-04-26T04:33:03.942 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.942 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.942 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:33:03.942 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821309+0000", 2024-04-26T04:33:03.942 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020262899999999999 2024-04-26T04:33:03.942 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.942 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.942 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:33:03.942 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.822530+0000", 2024-04-26T04:33:03.942 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0012206129999999999 2024-04-26T04:33:03.942 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.942 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.942 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316288+0000", 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.49375772400000001 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316375+0000", 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7312999999999996e-05 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316439+0000", 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3658999999999995e-05 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.943 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:161 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 45.176611645999998, 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51490446599999995, 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:33:03.944 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 161 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.945 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739292+0000", 2024-04-26T04:33:03.946 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6599999999999996e-07 2024-04-26T04:33:03.946 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.946 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.946 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.946 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739378+0000", 2024-04-26T04:33:03.946 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.5968000000000001e-05 2024-04-26T04:33:03.946 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.946 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.946 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.946 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739390+0000", 2024-04-26T04:33:03.946 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1595e-05 2024-04-26T04:33:03.946 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.946 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739398+0000", 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0029999999999994e-06 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.746760+0000", 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0073621420000000003 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.746876+0000", 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000115789 2024-04-26T04:33:03.947 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.747088+0000", 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021234800000000001 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252667+0000", 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.50557901500000002 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254031+0000", 2024-04-26T04:33:03.948 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001363518 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254124+0000", 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3376999999999997e-05 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254196+0000", 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2045000000000006e-05 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.949 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739429+0000", 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 45.176474462999998, 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.51514356699999997, 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 162 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.950 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739429+0000", 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739429+0000", 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739430+0000", 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8999999999999995e-07 2024-04-26T04:33:03.951 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.952 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.952 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.952 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739435+0000", 2024-04-26T04:33:03.952 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.5969999999999999e-06 2024-04-26T04:33:03.952 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.952 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.952 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.952 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739446+0000", 2024-04-26T04:33:03.953 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0535e-05 2024-04-26T04:33:03.953 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.953 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.953 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.953 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739453+0000", 2024-04-26T04:33:03.953 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.4800000000000004e-06 2024-04-26T04:33:03.953 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.953 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.953 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.943028+0000", 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20357512 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.943276+0000", 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024770199999999998 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.943640+0000", 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00036409399999999997 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.954 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252675+0000", 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30903449199999999 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254318+0000", 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016429179999999999 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254463+0000", 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000145623 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.955 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254573+0000", 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010921600000000001 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 45.176437802000002, 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.515346639, 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.956 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 163 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.957 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:33:03.958 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.958 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.958 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.958 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.958 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739466+0000", 2024-04-26T04:33:03.958 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.1699999999999997e-07 2024-04-26T04:33:03.958 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.958 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.958 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.958 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739471+0000", 2024-04-26T04:33:03.958 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.9219999999999996e-06 2024-04-26T04:33:03.958 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.958 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739487+0000", 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5435999999999999e-05 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739494+0000", 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2899999999999997e-06 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.944605+0000", 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20511091200000001 2024-04-26T04:33:03.959 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.944825+0000", 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00022035499999999999 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.945203+0000", 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00037760499999999999 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.252681+0000", 2024-04-26T04:33:03.960 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.307478311 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254680+0000", 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0019988129999999999 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254756+0000", 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5914999999999994e-05 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.961 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254812+0000", 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.6363000000000002e-05 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 45.176397665000003, 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.61583993999999997, 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:03.962 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 164 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:33:03.963 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0399999999999995e-07 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739594+0000", 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7046000000000007e-05 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739605+0000", 2024-04-26T04:33:03.964 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1576000000000001e-05 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739613+0000", 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1459999999999996e-06 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.046981+0000", 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30736738000000002 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.965 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.047229+0000", 2024-04-26T04:33:03.966 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024846999999999998 2024-04-26T04:33:03.966 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.966 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.966 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:33:03.966 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.047619+0000", 2024-04-26T04:33:03.966 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00038963799999999999 2024-04-26T04:33:03.966 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.966 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.966 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:33:03.966 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254219+0000", 2024-04-26T04:33:03.966 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20659982700000001 2024-04-26T04:33:03.966 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.966 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355048+0000", 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100829484 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355225+0000", 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017717399999999999 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355346+0000", 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000120495 2024-04-26T04:33:03.967 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 45.17627615, 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.71903434899999996, 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:33:03.968 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 165 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.969 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739628+0000", 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7000000000000004e-07 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739645+0000", 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6901999999999998e-05 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739656+0000", 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0556e-05 2024-04-26T04:33:03.970 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739667+0000", 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1535000000000001e-05 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355474+0000", 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.61580721299999996 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355612+0000", 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000137687 2024-04-26T04:33:03.971 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355816+0000", 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020451900000000001 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.357276+0000", 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0014599179999999999 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458392+0000", 2024-04-26T04:33:03.972 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101115921 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458547+0000", 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015461199999999999 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458662+0000", 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000114716 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.973 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:31 3.es0 3:7950d065:::smithi17829308-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.211346+0000", 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 38.704557002999998, 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80728777299999999, 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 31 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.974 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211346+0000", 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211346+0000", 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211350+0000", 2024-04-26T04:33:03.975 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.3400000000000002e-06 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211978+0000", 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00062828600000000004 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212008+0000", 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0287999999999999e-05 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.976 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212024+0000", 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5316e-05 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312296+0000", 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100272797 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312539+0000", 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000242542 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.977 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.313858+0000", 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001319312 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319757+0000", 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.005898927 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018515+0000", 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69875785199999996 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.978 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018634+0000", 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000119113 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:32 3.es0 3:7950d065:::smithi17829308-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.212058+0000", 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 38.703844994999997, 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80687460799999999, 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.979 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 32 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212058+0000", 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.980 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212058+0000", 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212060+0000", 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2130000000000001e-06 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212065+0000", 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.5260000000000001e-06 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.981 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.982 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.982 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212084+0000", 2024-04-26T04:33:03.982 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8399000000000001e-05 2024-04-26T04:33:03.982 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.982 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.982 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.982 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212114+0000", 2024-04-26T04:33:03.982 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0598000000000001e-05 2024-04-26T04:33:03.982 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.982 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.982 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.982 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.314776+0000", 2024-04-26T04:33:03.982 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10266165200000001 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.315053+0000", 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00027713999999999999 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316307+0000", 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00125442 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.983 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321719+0000", 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.005411463 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018816+0000", 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69709755699999998 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018933+0000", 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00011663999999999999 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.984 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:43 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~245760,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.217138+0000", 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 38.698764955999998, 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80275945999999998, 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 43 2024-04-26T04:33:03.985 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217138+0000", 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217138+0000", 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.986 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217141+0000", 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3499999999999999e-06 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217563+0000", 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00042241199999999999 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217591+0000", 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7501999999999999e-05 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.987 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217605+0000", 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3866000000000001e-05 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.317118+0000", 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.099513764000000005 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.317307+0000", 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018885199999999999 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.988 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.318924+0000", 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001616691 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321728+0000", 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0028038249999999998 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.019782+0000", 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69805386400000002 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.989 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.990 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.990 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.019898+0000", 2024-04-26T04:33:03.990 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000116334 2024-04-26T04:33:03.990 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.990 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.990 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.990 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.990 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.990 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:44 3.es0 3:7dd262f9:::smithi17829308-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:03.990 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.217633+0000", 2024-04-26T04:33:03.990 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 38.698270495000003, 2024-04-26T04:33:03.990 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80280496999999995, 2024-04-26T04:33:03.990 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.991 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.991 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:03.991 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.991 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:03.991 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:03.991 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 44 2024-04-26T04:33:03.991 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.991 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.991 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.991 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.991 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217633+0000", 2024-04-26T04:33:03.991 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.991 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217633+0000", 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217634+0000", 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.06e-06 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217640+0000", 2024-04-26T04:33:03.992 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.614e-06 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217654+0000", 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4279e-05 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217662+0000", 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2959999999999999e-06 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319858+0000", 2024-04-26T04:33:03.993 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102196092 2024-04-26T04:33:03.994 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.994 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.994 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.994 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319985+0000", 2024-04-26T04:33:03.994 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012646000000000001 2024-04-26T04:33:03.994 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.994 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.994 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:03.994 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.320976+0000", 2024-04-26T04:33:03.994 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00099148399999999994 2024-04-26T04:33:03.994 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.994 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.994 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.322066+0000", 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001090246 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.020239+0000", 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69817257799999999 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.020438+0000", 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00019886099999999999 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.995 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:267 3.2s0 3:4497087f:::smithi17829308-22:head [read 0~360448,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.131109+0000", 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 33.784794376999997, 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70452207899999997, 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 267 2024-04-26T04:33:03.996 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131109+0000", 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131109+0000", 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:03.997 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131111+0000", 2024-04-26T04:33:03.998 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9700000000000002e-06 2024-04-26T04:33:03.998 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.998 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.998 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:03.998 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131119+0000", 2024-04-26T04:33:03.998 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4600000000000003e-06 2024-04-26T04:33:03.998 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.998 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.998 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:03.998 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131136+0000", 2024-04-26T04:33:03.998 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6820000000000002e-05 2024-04-26T04:33:03.998 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.998 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131146+0000", 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.2499999999999995e-06 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131232+0000", 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6907e-05 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131352+0000", 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000119164 2024-04-26T04:33:03.999 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.835631+0000", 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70427950800000005 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:268 3.2s0 3:4497087f:::smithi17829308-22:head [read 0~360448,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 33.784734831000002, 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70524462600000004, 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:04.000 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 268 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.001 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:04.002 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:33:04.002 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:04.002 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.002 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.002 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:04.002 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:33:04.002 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3799999999999997e-07 2024-04-26T04:33:04.002 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.002 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.002 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:04.002 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131173+0000", 2024-04-26T04:33:04.002 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8929999999999998e-06 2024-04-26T04:33:04.002 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.003 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.003 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:04.003 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131188+0000", 2024-04-26T04:33:04.003 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5208000000000001e-05 2024-04-26T04:33:04.003 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.003 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.003 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:04.003 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131197+0000", 2024-04-26T04:33:04.003 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1950000000000003e-06 2024-04-26T04:33:04.003 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.003 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.003 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:04.003 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131969+0000", 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00077229900000000005 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.132116+0000", 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014686300000000001 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.836413+0000", 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70429752999999995 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:04.004 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:269 3.2s0 3:4497087f:::smithi17829308-22:head [read 0~360448,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.131236+0000", 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 33.784666940000001, 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70573893899999995, 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 269 2024-04-26T04:33:04.005 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131236+0000", 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131236+0000", 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:04.006 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131237+0000", 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.9800000000000003e-07 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131241+0000", 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.2239999999999997e-06 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131250+0000", 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.0629999999999995e-06 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.007 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231335+0000", 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10008484400000001 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231436+0000", 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000100435 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231576+0000", 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014048000000000001 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.008 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.836975+0000", 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.605399295 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:273 3.2s0 3:466bd692:::smithi17829308-25:head [delete] snapc 2=[2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.231373+0000", 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 33.684530015, 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60595961600000003, 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:04.009 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:04.010 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:04.010 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:04.010 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:04.010 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 273 2024-04-26T04:33:04.010 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.010 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:04.010 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.010 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:04.010 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231373+0000", 2024-04-26T04:33:04.010 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:04.010 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.010 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.010 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231373+0000", 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231375+0000", 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.539e-06 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231383+0000", 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5660000000000001e-06 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.011 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231394+0000", 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1739000000000001e-05 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231405+0000", 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0266e-05 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231995+0000", 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00059051299999999998 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.012 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.232097+0000", 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010197899999999999 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.233122+0000", 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00102451 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.234316+0000", 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0011942019999999999 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.013 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.837219+0000", 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60290286500000001 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.837333+0000", 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000114437 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.014 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:349 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~442368] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.325227+0000", 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 28.590676143, 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.56835219299999995, 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 349 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.015 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.325227+0000", 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.325227+0000", 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.325231+0000", 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8500000000000004e-06 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.016 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.017 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:04.017 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.325808+0000", 2024-04-26T04:33:04.017 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00057670400000000004 2024-04-26T04:33:04.017 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.017 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.017 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:04.017 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.325852+0000", 2024-04-26T04:33:04.017 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.3961000000000001e-05 2024-04-26T04:33:04.017 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.017 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.017 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:04.017 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.426013+0000", 2024-04-26T04:33:04.017 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100160764 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.426151+0000", 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013824700000000001 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op must wait for map", 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.426163+0000", 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2656e-05 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:04.018 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.580183+0000", 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.154019447 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.580443+0000", 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00025980799999999999 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.582459+0000", 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0020159050000000001 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:04.019 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.590550+0000", 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0080916990000000008 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.893438+0000", 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.302887772 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.893579+0000", 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014138000000000001 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:04.020 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:351 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~49152] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.427267+0000", 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 28.488636240000002, 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.56700841899999999, 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 351 2024-04-26T04:33:04.021 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.427267+0000", 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.427267+0000", 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:04.022 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.427269+0000", 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5960000000000001e-06 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.427330+0000", 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.1243999999999998e-05 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.427353+0000", 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3393000000000001e-05 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:04.023 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.527490+0000", 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100136091 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.587991+0000", 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.060501006000000003 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.588318+0000", 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000327125 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:04.024 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.589552+0000", 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001233998 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.591162+0000", 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016098900000000001 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.994158+0000", 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.402995995 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.025 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.994276+0000", 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000118081 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:353 3.es0 3:7dd262f9:::smithi17829308-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.627803+0000", 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 28.28810086, 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.66700178399999999, 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:04.026 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 353 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627803+0000", 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627803+0000", 2024-04-26T04:33:04.027 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627804+0000", 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.68e-06 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627813+0000", 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4759999999999998e-06 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627833+0000", 2024-04-26T04:33:04.028 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9809e-05 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627844+0000", 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1250000000000001e-05 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.691576+0000", 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.063732043000000002 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.691730+0000", 2024-04-26T04:33:04.029 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015389900000000001 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.995375+0000", 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30364495600000002 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.995640+0000", 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026531100000000001 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:04.030 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:36.294804+0000", 2024-04-26T04:33:04.031 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.29916436000000002 2024-04-26T04:33:04.031 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.031 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:04.031 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.031 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:04.031 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:04.031 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:04.032 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T04:33:04.033 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:04.030+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:04.033 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T04:33:04.134 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:04.132+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:04.164 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:04.164 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:04.164 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:33:04.164 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:04.178 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T04:33:04.232 INFO:tasks.rados.rados.0.smithi178.stdout:297: done (3 left) 2024-04-26T04:33:04.232 INFO:tasks.rados.rados.0.smithi178.stdout:298: snap_create 2024-04-26T04:33:04.232 INFO:tasks.rados.rados.0.smithi178.stdout:289: finishing write tid 1 to smithi17829308-26 2024-04-26T04:33:04.232 INFO:tasks.rados.rados.0.smithi178.stdout:289: finishing write tid 2 to smithi17829308-26 2024-04-26T04:33:04.232 INFO:tasks.rados.rados.0.smithi178.stdout:289: finishing write tid 3 to smithi17829308-26 2024-04-26T04:33:04.232 INFO:tasks.rados.rados.0.smithi178.stdout:289: finishing write tid 4 to smithi17829308-26 2024-04-26T04:33:04.232 INFO:tasks.rados.rados.0.smithi178.stdout:289: finishing write tid 5 to smithi17829308-26 2024-04-26T04:33:04.232 INFO:tasks.rados.rados.0.smithi178.stdout:289: finishing write tid 6 to smithi17829308-26 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:289: oid 26 updating version 0 to 56 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:289: oid 26 updating version 56 to 57 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:289: oid 26 updating version 57 to 59 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:289: oid 26 version 59 is already newer than 58 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:289: oid 26 version 59 is already newer than 55 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 26 v 59 (ObjNum 120 snap 16 seq_num 120) dirty exists 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:289: left oid 26 (ObjNum 120 snap 16 seq_num 120) 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:292: finishing write tid 7 to smithi17829308-20 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:292: oid 20 updating version 0 to 45 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:292: oid 20 updating version 45 to 46 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:292: oid 20 version 46 is already newer than 43 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:292: oid 20 version 46 is already newer than 41 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:292: oid 20 version 46 is already newer than 44 2024-04-26T04:33:04.233 INFO:tasks.rados.rados.0.smithi178.stdout:292: oid 20 version 46 is already newer than 42 2024-04-26T04:33:04.234 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 20 v 46 (ObjNum 122 snap 16 seq_num 122) dirty exists 2024-04-26T04:33:04.234 INFO:tasks.rados.rados.0.smithi178.stdout:292: left oid 20 (ObjNum 122 snap 16 seq_num 122) 2024-04-26T04:33:04.235 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:04.233+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:04.242 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:33:04.242 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:33:04.260 INFO:tasks.rados.rados.0.smithi178.stdout:296: finishing copy_from racing read to smithi17829308-32 2024-04-26T04:33:04.271 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:33:04.271 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:33:04.291 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:33:04.291 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:33:04.313 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:33:04.313 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:33:04.321 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-26T04:33:04.332 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:33:04.332 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:33:04.336 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:04.333+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:04.337 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:04.334+0000 7fd32cbef640 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap b after trimming was started 2024-04-26T04:33:04.350 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:04.350 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:04.354 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:33:04.354 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:33:04.365 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T04:33:04.376 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:33:04.376 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:33:04.400 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:33:04.400 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:33:04.418 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:33:04.419 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:33:04.436 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:04.434+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:04.484 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:33:04.485 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:33:04.489 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:04.489 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:04.504 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T04:33:04.504 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:33:04.505 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:33:04.537 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:04.537+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:04.566 INFO:tasks.rados.rados.0.smithi178.stdout:296: finishing copy_from to smithi17829308-32 2024-04-26T04:33:04.567 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 32 v 44 (ObjNum 87 snap 12 seq_num 87) dirty exists 2024-04-26T04:33:04.595 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:33:04.595 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:33:04.612 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:33:04.613 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:33:04.629 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:04.629 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:04.638 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:04.636+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:04.643 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T04:33:04.668 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:33:04.668 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:33:04.699 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:33:04.699 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:33:04.726 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:33:04.727 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:33:04.739 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:04.736+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:04.810 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:04.811 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:04.826 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T04:33:04.839 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:04.838+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:04.941 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:04.941+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:05.001 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:05.001 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:05.016 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T04:33:05.062 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:05.060+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:05.139 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:05.140 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:05.143 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:05.141+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:05.155 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T04:33:05.233 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-04-26T04:33:05.239 INFO:tasks.rados.rados.0.smithi178.stdout:289: done (3 left) 2024-04-26T04:33:05.239 INFO:tasks.rados.rados.0.smithi178.stdout:292: done (2 left) 2024-04-26T04:33:05.239 INFO:tasks.rados.rados.0.smithi178.stdout:296: done (1 left) 2024-04-26T04:33:05.239 INFO:tasks.rados.rados.0.smithi178.stdout:298: done (0 left) 2024-04-26T04:33:05.239 INFO:tasks.rados.rados.0.smithi178.stdout:299: read oid 15 snap -1 2024-04-26T04:33:05.240 INFO:tasks.rados.rados.0.smithi178.stdout:299: expect (ObjNum 115 snap 16 seq_num 115) 2024-04-26T04:33:05.240 INFO:tasks.rados.rados.0.smithi178.stdout:300: copy_from oid 3 from oid 47 current snap is 17 2024-04-26T04:33:05.240 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 38 current snap is 17 2024-04-26T04:33:05.240 INFO:tasks.rados.rados.0.smithi178.stdout:301: seq_num 124 ranges {65536=540672,606208=720896,1327104=786432,2113536=49152} 2024-04-26T04:33:05.244 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:05.242+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:05.249 INFO:tasks.rados.rados.0.smithi178.stdout:301: writing smithi17829308-38 from 65536 to 606208 tid 1 2024-04-26T04:33:05.262 INFO:tasks.rados.rados.0.smithi178.stdout:301: writing smithi17829308-38 from 606208 to 1327104 tid 2 2024-04-26T04:33:05.274 INFO:tasks.rados.rados.0.smithi178.stdout:301: writing smithi17829308-38 from 1327104 to 2113536 tid 3 2024-04-26T04:33:05.275 INFO:tasks.rados.rados.0.smithi178.stdout:301: writing smithi17829308-38 from 2113536 to 2162688 tid 4 2024-04-26T04:33:05.275 INFO:tasks.rados.rados.0.smithi178.stdout:302: read oid 39 snap -1 2024-04-26T04:33:05.275 INFO:tasks.rados.rados.0.smithi178.stdout:302: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:33:05.275 INFO:tasks.rados.rados.0.smithi178.stdout:303: rmattr oid 46 current snap is 17 2024-04-26T04:33:05.275 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 33 current snap is 17 2024-04-26T04:33:05.276 INFO:tasks.rados.rados.0.smithi178.stdout:304: seq_num 125 ranges {4423680=557056,4980736=475136,5455872=507904,5963776=475136,6438912=98304} 2024-04-26T04:33:05.325 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:05.325 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:05.329 INFO:tasks.rados.rados.0.smithi178.stdout:304: writing smithi17829308-33 from 4423680 to 4980736 tid 1 2024-04-26T04:33:05.333 INFO:tasks.rados.rados.0.smithi178.stdout:304: writing smithi17829308-33 from 4980736 to 5455872 tid 2 2024-04-26T04:33:05.337 INFO:tasks.rados.rados.0.smithi178.stdout:304: writing smithi17829308-33 from 5455872 to 5963776 tid 3 2024-04-26T04:33:05.339 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T04:33:05.341 INFO:tasks.rados.rados.0.smithi178.stdout:304: writing smithi17829308-33 from 5963776 to 6438912 tid 4 2024-04-26T04:33:05.343 INFO:tasks.rados.rados.0.smithi178.stdout:304: writing smithi17829308-33 from 6438912 to 6537216 tid 5 2024-04-26T04:33:05.343 INFO:tasks.rados.rados.0.smithi178.stdout:305: read oid 2 snap -1 2024-04-26T04:33:05.343 INFO:tasks.rados.rados.0.smithi178.stdout:305: expect (ObjNum 88 snap 12 seq_num 88) 2024-04-26T04:33:05.343 INFO:tasks.rados.rados.0.smithi178.stdout:306: copy_from oid 23 from oid 43 current snap is 17 2024-04-26T04:33:05.343 INFO:tasks.rados.rados.0.smithi178.stdout:307: read oid 7 snap -1 2024-04-26T04:33:05.344 INFO:tasks.rados.rados.0.smithi178.stdout:307: expect deleted 2024-04-26T04:33:05.344 INFO:tasks.rados.rados.0.smithi178.stdout:308: read oid 27 snap 16 2024-04-26T04:33:05.344 INFO:tasks.rados.rados.0.smithi178.stdout:308: expect (ObjNum 87 snap 12 seq_num 87) 2024-04-26T04:33:05.344 INFO:tasks.rados.rados.0.smithi178.stdout:309: read oid 45 snap -1 2024-04-26T04:33:05.344 INFO:tasks.rados.rados.0.smithi178.stdout:309: expect (ObjNum 79 snap 8 seq_num 79) 2024-04-26T04:33:05.344 INFO:tasks.rados.rados.0.smithi178.stdout:310: copy_from oid 31 from oid 41 current snap is 17 2024-04-26T04:33:05.344 INFO:tasks.rados.rados.0.smithi178.stdout:311: snap_create 2024-04-26T04:33:05.386 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:05.384+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:05.447 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:05.442+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:05.458 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:05.458 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:05.472 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T04:33:05.547 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:05.545+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:05.558 INFO:tasks.rados.rados.0.smithi178.stdout:300: finishing copy_from to smithi17829308-3 2024-04-26T04:33:05.558 INFO:tasks.rados.rados.0.smithi178.stdout:300: got expected ENOENT (src dne) 2024-04-26T04:33:05.558 INFO:tasks.rados.rados.0.smithi178.stdout:300: finishing copy_from racing read to smithi17829308-3 2024-04-26T04:33:05.642 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:05.642 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:05.647 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:05.645+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:05.654 INFO:tasks.rados.rados.0.smithi178.stdout:309: expect (ObjNum 79 snap 8 seq_num 79) 2024-04-26T04:33:05.657 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T04:33:05.737 INFO:tasks.rados.rados.0.smithi178.stdout:306: finishing copy_from racing read to smithi17829308-23 2024-04-26T04:33:05.737 INFO:tasks.rados.rados.0.smithi178.stdout:302: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:33:05.749 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:05.745+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:05.758 INFO:tasks.rados.rados.0.smithi178.stdout:310: finishing copy_from to smithi17829308-31 2024-04-26T04:33:05.759 INFO:tasks.rados.rados.0.smithi178.stdout:310: got expected ENOENT (src dne) 2024-04-26T04:33:05.759 INFO:tasks.rados.rados.0.smithi178.stdout:310: finishing copy_from racing read to smithi17829308-31 2024-04-26T04:33:05.762 INFO:tasks.rados.rados.0.smithi178.stdout:301: finishing write tid 1 to smithi17829308-38 2024-04-26T04:33:05.762 INFO:tasks.rados.rados.0.smithi178.stdout:301: finishing write tid 2 to smithi17829308-38 2024-04-26T04:33:05.762 INFO:tasks.rados.rados.0.smithi178.stdout:301: finishing write tid 3 to smithi17829308-38 2024-04-26T04:33:05.766 INFO:tasks.rados.rados.0.smithi178.stdout:299: expect (ObjNum 115 snap 16 seq_num 115) 2024-04-26T04:33:05.777 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:05.777 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:05.791 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T04:33:05.850 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:05.849+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:05.914 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:05.928 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T04:33:05.975 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:05.974+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:06.032 INFO:tasks.rados.rados.0.smithi178.stdout:305: expect (ObjNum 88 snap 12 seq_num 88) 2024-04-26T04:33:06.051 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:06.049+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:06.051 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:06.052 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:06.065 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T04:33:06.153 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:06.149+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:06.192 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [2] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T04:33:06.193 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T04:33:06.193 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-04-26T04:33:06.193 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:06.239 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T04:33:06.253 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:06.248+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:06.284 INFO:tasks.rados.rados.0.smithi178.stdout:304: finishing write tid 1 to smithi17829308-33 2024-04-26T04:33:06.284 INFO:tasks.rados.rados.0.smithi178.stdout:308: expect (ObjNum 87 snap 12 seq_num 87) 2024-04-26T04:33:06.354 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:06.353+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:06.357 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:06.357 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:06.372 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T04:33:06.377 INFO:tasks.rados.rados.0.smithi178.stdout:304: finishing write tid 2 to smithi17829308-33 2024-04-26T04:33:06.377 INFO:tasks.rados.rados.0.smithi178.stdout:306: finishing copy_from to smithi17829308-23 2024-04-26T04:33:06.377 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 23 v 38 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-04-26T04:33:06.377 INFO:tasks.rados.rados.0.smithi178.stdout:299: done (12 left) 2024-04-26T04:33:06.377 INFO:tasks.rados.rados.0.smithi178.stdout:300: done (11 left) 2024-04-26T04:33:06.377 INFO:tasks.rados.rados.0.smithi178.stdout:302: done (10 left) 2024-04-26T04:33:06.377 INFO:tasks.rados.rados.0.smithi178.stdout:305: done (9 left) 2024-04-26T04:33:06.377 INFO:tasks.rados.rados.0.smithi178.stdout:306: done (8 left) 2024-04-26T04:33:06.377 INFO:tasks.rados.rados.0.smithi178.stdout:307: done (7 left) 2024-04-26T04:33:06.378 INFO:tasks.rados.rados.0.smithi178.stdout:308: done (6 left) 2024-04-26T04:33:06.378 INFO:tasks.rados.rados.0.smithi178.stdout:309: done (5 left) 2024-04-26T04:33:06.378 INFO:tasks.rados.rados.0.smithi178.stdout:310: done (4 left) 2024-04-26T04:33:06.378 INFO:tasks.rados.rados.0.smithi178.stdout:311: done (3 left) 2024-04-26T04:33:06.378 INFO:tasks.rados.rados.0.smithi178.stdout:312: delete oid 45 current snap is 18 2024-04-26T04:33:06.378 INFO:tasks.rados.rados.0.smithi178.stdout:304: finishing write tid 3 to smithi17829308-33 2024-04-26T04:33:06.378 INFO:tasks.rados.rados.0.smithi178.stdout:304: finishing write tid 4 to smithi17829308-33 2024-04-26T04:33:06.378 INFO:tasks.rados.rados.0.smithi178.stdout:304: finishing write tid 5 to smithi17829308-33 2024-04-26T04:33:06.378 INFO:tasks.rados.rados.0.smithi178.stdout:304: finishing write tid 6 to smithi17829308-33 2024-04-26T04:33:06.378 INFO:tasks.rados.rados.0.smithi178.stdout:301: finishing write tid 4 to smithi17829308-38 2024-04-26T04:33:06.379 INFO:tasks.rados.rados.0.smithi178.stdout:301: finishing write tid 5 to smithi17829308-38 2024-04-26T04:33:06.441 INFO:tasks.rados.rados.0.smithi178.stdout:304: finishing write tid 7 to smithi17829308-33 2024-04-26T04:33:06.441 INFO:tasks.rados.rados.0.smithi178.stdout:304: oid 33 updating version 0 to 53 2024-04-26T04:33:06.441 INFO:tasks.rados.rados.0.smithi178.stdout:304: oid 33 updating version 53 to 54 2024-04-26T04:33:06.441 INFO:tasks.rados.rados.0.smithi178.stdout:304: oid 33 updating version 54 to 56 2024-04-26T04:33:06.441 INFO:tasks.rados.rados.0.smithi178.stdout:304: oid 33 updating version 56 to 57 2024-04-26T04:33:06.442 INFO:tasks.rados.rados.0.smithi178.stdout:304: oid 33 updating version 57 to 58 2024-04-26T04:33:06.442 INFO:tasks.rados.rados.0.smithi178.stdout:304: oid 33 version 58 is already newer than 55 2024-04-26T04:33:06.442 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 33 v 58 (ObjNum 125 snap 17 seq_num 125) dirty exists 2024-04-26T04:33:06.442 INFO:tasks.rados.rados.0.smithi178.stdout:304: left oid 33 (ObjNum 125 snap 17 seq_num 125) 2024-04-26T04:33:06.455 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:06.454+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:06.501 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:06.501 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:06.515 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T04:33:06.516 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:33:06.516 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:06.516 INFO:teuthology.orchestra.run.smithi012.stdout: "epoch": 71, 2024-04-26T04:33:06.517 INFO:teuthology.orchestra.run.smithi012.stdout: "fsid": "12c27fbe-0161-48d0-88ec-42a226685437", 2024-04-26T04:33:06.517 INFO:teuthology.orchestra.run.smithi012.stdout: "created": "2024-04-26T04:31:56.451062+0000", 2024-04-26T04:33:06.517 INFO:teuthology.orchestra.run.smithi012.stdout: "modified": "2024-04-26T04:33:06.233796+0000", 2024-04-26T04:33:06.517 INFO:teuthology.orchestra.run.smithi012.stdout: "last_up_change": "2024-04-26T04:32:09.452464+0000", 2024-04-26T04:33:06.517 INFO:teuthology.orchestra.run.smithi012.stdout: "last_in_change": "2024-04-26T04:32:20.674579+0000", 2024-04-26T04:33:06.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-26T04:33:06.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_num": 5799936, 2024-04-26T04:33:06.517 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_set": [ 2024-04-26T04:33:06.517 INFO:teuthology.orchestra.run.smithi012.stdout: "pglog_hardlimit", 2024-04-26T04:33:06.517 INFO:teuthology.orchestra.run.smithi012.stdout: "purged_snapdirs", 2024-04-26T04:33:06.517 INFO:teuthology.orchestra.run.smithi012.stdout: "recovery_deletes", 2024-04-26T04:33:06.517 INFO:teuthology.orchestra.run.smithi012.stdout: "sortbitwise" 2024-04-26T04:33:06.518 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:33:06.518 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_version": 6, 2024-04-26T04:33:06.518 INFO:teuthology.orchestra.run.smithi012.stdout: "full_ratio": 0.94999998807907104, 2024-04-26T04:33:06.518 INFO:teuthology.orchestra.run.smithi012.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-26T04:33:06.518 INFO:teuthology.orchestra.run.smithi012.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-26T04:33:06.518 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_snapshot": "", 2024-04-26T04:33:06.518 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_max": 3, 2024-04-26T04:33:06.518 INFO:teuthology.orchestra.run.smithi012.stdout: "max_osd": 16, 2024-04-26T04:33:06.518 INFO:teuthology.orchestra.run.smithi012.stdout: "require_min_compat_client": "luminous", 2024-04-26T04:33:06.518 INFO:teuthology.orchestra.run.smithi012.stdout: "min_compat_client": "luminous", 2024-04-26T04:33:06.518 INFO:teuthology.orchestra.run.smithi012.stdout: "require_osd_release": "squid", 2024-04-26T04:33:06.518 INFO:teuthology.orchestra.run.smithi012.stdout: "allow_crimson": false, 2024-04-26T04:33:06.519 INFO:teuthology.orchestra.run.smithi012.stdout: "pools": [ 2024-04-26T04:33:06.519 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.519 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 1, 2024-04-26T04:33:06.519 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": ".mgr", 2024-04-26T04:33:06.519 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-04-26T04:32:07.426875+0000", 2024-04-26T04:33:06.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T04:33:06.519 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool", 2024-04-26T04:33:06.519 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-04-26T04:33:06.519 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-04-26T04:33:06.519 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-04-26T04:33:06.519 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-04-26T04:33:06.519 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-04-26T04:33:06.520 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-04-26T04:33:06.520 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T04:33:06.520 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T04:33:06.520 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-04-26T04:33:06.520 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-04-26T04:33:06.520 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 1, 2024-04-26T04:33:06.520 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 1, 2024-04-26T04:33:06.520 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 1, 2024-04-26T04:33:06.520 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 1, 2024-04-26T04:33:06.520 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 1, 2024-04-26T04:33:06.520 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-04-26T04:33:06.521 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-04-26T04:33:06.521 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-04-26T04:33:06.521 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-04-26T04:33:06.521 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-04-26T04:33:06.521 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-04-26T04:33:06.521 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-04-26T04:33:06.521 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.521 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "25", 2024-04-26T04:33:06.521 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-04-26T04:33:06.521 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-26T04:33:06.521 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-04-26T04:33:06.521 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-04-26T04:33:06.522 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-04-26T04:33:06.522 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 0, 2024-04-26T04:33:06.522 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 0, 2024-04-26T04:33:06.522 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-04-26T04:33:06.522 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-04-26T04:33:06.522 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-04-26T04:33:06.522 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-04-26T04:33:06.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-04-26T04:33:06.522 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-04-26T04:33:06.523 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-04-26T04:33:06.523 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-04-26T04:33:06.523 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-04-26T04:33:06.523 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-04-26T04:33:06.523 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-04-26T04:33:06.523 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T04:33:06.524 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T04:33:06.524 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T04:33:06.524 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-04-26T04:33:06.524 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-04-26T04:33:06.524 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-04-26T04:33:06.525 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-04-26T04:33:06.526 INFO:teuthology.orchestra.run.smithi012.stdout: "options": { 2024-04-26T04:33:06.526 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_max": 32, 2024-04-26T04:33:06.526 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_min": 1 2024-04-26T04:33:06.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.526 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-04-26T04:33:06.526 INFO:teuthology.orchestra.run.smithi012.stdout: "mgr": {} 2024-04-26T04:33:06.526 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.526 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-04-26T04:33:06.526 INFO:teuthology.orchestra.run.smithi012.stdout: "score_type": "Fair distribution", 2024-04-26T04:33:06.526 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 15.380000114440918, 2024-04-26T04:33:06.526 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 15.380000114440918, 2024-04-26T04:33:06.526 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.12999999523162842, 2024-04-26T04:33:06.526 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2, 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2, 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.96524238586425781, 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 2, 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "rbd", 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-04-26T04:32:11.318957+0000", 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 1, 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool", 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 1, 2024-04-26T04:33:06.527 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 2, 2024-04-26T04:33:06.528 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 1, 2024-04-26T04:33:06.528 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 0, 2024-04-26T04:33:06.528 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-04-26T04:33:06.528 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-04-26T04:33:06.528 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T04:33:06.528 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T04:33:06.528 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-04-26T04:33:06.528 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-04-26T04:33:06.528 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 8, 2024-04-26T04:33:06.528 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 8, 2024-04-26T04:33:06.528 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 8, 2024-04-26T04:33:06.528 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 8, 2024-04-26T04:33:06.528 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 8, 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "27", 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-04-26T04:33:06.529 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-04-26T04:33:06.530 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 0, 2024-04-26T04:33:06.530 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 0, 2024-04-26T04:33:06.530 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-04-26T04:33:06.530 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-04-26T04:33:06.530 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-04-26T04:33:06.530 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-04-26T04:33:06.530 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-04-26T04:33:06.530 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-04-26T04:33:06.530 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-04-26T04:33:06.530 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-04-26T04:33:06.530 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-04-26T04:33:06.530 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-04-26T04:33:06.530 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "", 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-04-26T04:33:06.531 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 0, 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: "rbd": {} 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: "read_balance": { 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: "score_type": "Fair distribution", 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: "score_acting": 4.0199999809265137, 2024-04-26T04:33:06.532 INFO:teuthology.orchestra.run.smithi012.stdout: "score_stable": 4.0199999809265137, 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: "optimal_score": 0.56000000238418579, 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_acting": 2.25, 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: "raw_score_stable": 2.25, 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity_weighted": 0.96524238586425781, 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity": 0.97000002861022949, 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: "average_primary_affinity_weighted": 0.97000002861022949 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_name": "unique_pool_0", 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: "create_time": "2024-04-26T04:32:21.196824+0000", 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flags": 8193, 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-26T04:33:06.533 INFO:teuthology.orchestra.run.smithi012.stdout: "type": 3, 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 7, 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "min_size": 5, 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_rule": 1, 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_count": 0, 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_target": 0, 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_barrier": 0, 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "object_hash": 2, 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_autoscale_mode": "off", 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num": 16, 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num": 16, 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_placement_num_target": 16, 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_target": 16, 2024-04-26T04:33:06.534 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_num_pending": 16, 2024-04-26T04:33:06.535 INFO:teuthology.orchestra.run.smithi012.stdout: "last_pg_merge_meta": { 2024-04-26T04:33:06.535 INFO:teuthology.orchestra.run.smithi012.stdout: "source_pgid": "0.0", 2024-04-26T04:33:06.535 INFO:teuthology.orchestra.run.smithi012.stdout: "ready_epoch": 0, 2024-04-26T04:33:06.535 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_started": 0, 2024-04-26T04:33:06.535 INFO:teuthology.orchestra.run.smithi012.stdout: "last_epoch_clean": 0, 2024-04-26T04:33:06.535 INFO:teuthology.orchestra.run.smithi012.stdout: "source_version": "0'0", 2024-04-26T04:33:06.535 INFO:teuthology.orchestra.run.smithi012.stdout: "target_version": "0'0" 2024-04-26T04:33:06.535 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.535 INFO:teuthology.orchestra.run.smithi012.stdout: "last_change": "71", 2024-04-26T04:33:06.535 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend": "0", 2024-04-26T04:33:06.535 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-26T04:33:06.535 INFO:teuthology.orchestra.run.smithi012.stdout: "last_force_op_resend_preluminous": "0", 2024-04-26T04:33:06.535 INFO:teuthology.orchestra.run.smithi012.stdout: "auid": 0, 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_mode": "selfmanaged", 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_seq": 19, 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "snap_epoch": 71, 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "pool_snaps": [], 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps": "[]", 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_bytes": 0, 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "quota_max_objects": 0, 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "tiers": [], 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "tier_of": -1, 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "read_tier": -1, 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "write_tier": -1, 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_mode": "none", 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_bytes": 0, 2024-04-26T04:33:06.536 INFO:teuthology.orchestra.run.smithi012.stdout: "target_max_objects": 0, 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_flush_age": 0, 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: "cache_min_evict_age": 0, 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profile": "shecprofile", 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_params": { 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "none" 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_period": 0, 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_count": 0, 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: "use_gmt_hitset": true, 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: "min_read_recency_for_promote": 0, 2024-04-26T04:33:06.537 INFO:teuthology.orchestra.run.smithi012.stdout: "min_write_recency_for_promote": 0, 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_grade_decay_rate": 0, 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: "hit_set_search_last_n": 0, 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: "grade_table": [], 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: "stripe_width": 16384, 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: "expected_num_objects": 0, 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: "fast_read": false, 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: "options": {}, 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: "application_metadata": { 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: "rados": {} 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: "osds": [ 2024-04-26T04:33:06.538 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "55a4f254-2c2a-4007-9c52-b89df6fffb69", 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 21, 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.539 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6816", 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6817", 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.540 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6818", 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6819", 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.541 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6822", 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6823", 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.542 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6820", 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6821", 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1754063127 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6817/1754063127", 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6819/1754063127", 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6823/1754063127", 2024-04-26T04:33:06.543 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6821/1754063127", 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "32db4f97-c8ac-481c-9e80-18fc91004298", 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.544 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 0, 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6808", 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6809", 2024-04-26T04:33:06.545 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6810", 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6811", 2024-04-26T04:33:06.546 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6814", 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6817", 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:33:06.547 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6812", 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6813", 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3264326676 2024-04-26T04:33:06.548 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.113:6809/3264326676", 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.113:6811/3264326676", 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.113:6817/3264326676", 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.113:6813/3264326676", 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-04-26T04:33:06.549 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "2cd26466-5d59-4599-ba3f-a78b0e3b681f", 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 0, 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 0, 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 0, 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.550 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6800", 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6802", 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.551 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6804", 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6806", 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.552 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6812", 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6814", 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.553 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6808", 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6810", 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454475448 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.137:6802/1454475448", 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.137:6806/1454475448", 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.137:6814/1454475448", 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.137:6810/1454475448", 2024-04-26T04:33:06.554 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "0945017d-9525-49d6-aa67-6635acd94c3e", 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.555 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 22, 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 60, 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6803", 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6805", 2024-04-26T04:33:06.556 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6808", 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6810", 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:33:06.557 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.558 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.558 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.558 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.558 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.558 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.558 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6816", 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6817", 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.559 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6813", 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6815", 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2898698985 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.178:6805/2898698985", 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.178:6810/2898698985", 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.178:6817/2898698985", 2024-04-26T04:33:06.560 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.178:6815/2898698985", 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "38dc68bf-0a89-4900-9d35-24c70368447a", 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.561 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.562 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 21, 2024-04-26T04:33:06.562 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:33:06.562 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.562 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.562 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.562 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.562 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.562 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.562 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6808", 2024-04-26T04:33:06.562 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:33:06.562 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.563 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.563 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6809", 2024-04-26T04:33:06.563 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:33:06.563 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.563 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.563 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.563 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.563 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.563 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.563 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.563 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6810", 2024-04-26T04:33:06.563 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6811", 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6814", 2024-04-26T04:33:06.564 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6815", 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6812", 2024-04-26T04:33:06.565 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6813", 2024-04-26T04:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3209840823 2024-04-26T04:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6809/3209840823", 2024-04-26T04:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6811/3209840823", 2024-04-26T04:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6815/3209840823", 2024-04-26T04:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6813/3209840823", 2024-04-26T04:33:06.566 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "e91cc8a1-57e1-4760-a329-8a60aa752a95", 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:33:06.567 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 60, 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6800", 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6801", 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.568 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6802", 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6803", 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.569 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6806", 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6807", 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.570 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6804", 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6805", 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1475637773 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.571 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.113:6801/1475637773", 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.113:6803/1475637773", 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.113:6807/1475637773", 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.113:6805/1475637773", 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 6, 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "a1322130-3391-432d-a1ee-f83297885029", 2024-04-26T04:33:06.572 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 25, 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6816", 2024-04-26T04:33:06.573 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:33:06.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.574 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.574 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6817", 2024-04-26T04:33:06.574 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:33:06.574 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.574 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.574 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.574 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6818", 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6819", 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.575 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6822", 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6823", 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.576 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6820", 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6821", 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 771338783 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.137:6817/771338783", 2024-04-26T04:33:06.577 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.137:6819/771338783", 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.137:6823/771338783", 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.137:6821/771338783", 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 7, 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "02ee4ca2-9d48-4aff-9641-1823760c7a3b", 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.578 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 22, 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6826", 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6827", 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6828", 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6829", 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6832", 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.581 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6833", 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6830", 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6831", 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1017152077 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.178:6827/1017152077", 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.178:6829/1017152077", 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.178:6833/1017152077", 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.178:6831/1017152077", 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 8, 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "ea3d5f62-3d68-4824-8556-da8db36553fb", 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 21, 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6800", 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6801", 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6802", 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6803", 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6806", 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6807", 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6804", 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6805", 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2153129138 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6801/2153129138", 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6803/2153129138", 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6807/2153129138", 2024-04-26T04:33:06.588 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6805/2153129138", 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 9, 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70", 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.589 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.590 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:33:06.590 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 0, 2024-04-26T04:33:06.590 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.590 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.590 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.590 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.590 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.590 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6824", 2024-04-26T04:33:06.590 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:33:06.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.590 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.591 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6825", 2024-04-26T04:33:06.591 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:33:06.591 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.591 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.591 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.591 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.591 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.591 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6826", 2024-04-26T04:33:06.591 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6827", 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6830", 2024-04-26T04:33:06.592 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6831", 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6828", 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:33:06.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6829", 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 359931551 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.113:6825/359931551", 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.113:6827/359931551", 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.113:6831/359931551", 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.113:6829/359931551", 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.594 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 10, 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "8c4f8df5-465a-486e-9413-1de14e33c119", 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 21, 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 60, 2024-04-26T04:33:06.595 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6801", 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6803", 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.596 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6805", 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6807", 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6813", 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6815", 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.598 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.599 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.599 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6809", 2024-04-26T04:33:06.599 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:33:06.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.599 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.599 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6811", 2024-04-26T04:33:06.599 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1533480861 2024-04-26T04:33:06.599 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.599 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.137:6803/1533480861", 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.137:6807/1533480861", 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.137:6815/1533480861", 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.137:6811/1533480861", 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 11, 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "6fe54c2d-0b95-4235-8c22-51230cfaa591", 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.600 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 22, 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6818", 2024-04-26T04:33:06.601 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6819", 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6820", 2024-04-26T04:33:06.602 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6821", 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6824", 2024-04-26T04:33:06.603 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6825", 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6822", 2024-04-26T04:33:06.604 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6823", 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1683682957 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.178:6819/1683682957", 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.178:6821/1683682957", 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.178:6825/1683682957", 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.178:6823/1683682957", 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.605 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 12, 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "5ddd0f05-fb78-43bb-95ec-3fc7985603b3", 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 0, 2024-04-26T04:33:06.606 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6824", 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6825", 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.607 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6826", 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6827", 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.608 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6830", 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6831", 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6828", 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.12:6829", 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 2509311840 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.12:6825/2509311840", 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.12:6827/2509311840", 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.12:6831/2509311840", 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.12:6829/2509311840", 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 13, 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "37eb42c1-f29e-49f5-b8f6-035178038e5d", 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.611 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 0.443878173828125, 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 31, 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6815", 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:33:06.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6816", 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6818", 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6819", 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6822", 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6823", 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6820", 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.113:6821", 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 3716239663 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.113:6816/3716239663", 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.113:6819/3716239663", 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.113:6823/3716239663", 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.113:6821/3716239663", 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 14, 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "b2267367-a857-4a5c-a8d1-6a9b23c748f6", 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 23, 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.617 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.618 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.618 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.618 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6824", 2024-04-26T04:33:06.618 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:33:06.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.618 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.618 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6825", 2024-04-26T04:33:06.618 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:33:06.618 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.618 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6826", 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6827", 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.619 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6830", 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6831", 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.620 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6828", 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.137:6829", 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 1454956132 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.137:6825/1454956132", 2024-04-26T04:33:06.621 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.137:6827/1454956132", 2024-04-26T04:33:06.622 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.137:6831/1454956132", 2024-04-26T04:33:06.622 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.137:6829/1454956132", 2024-04-26T04:33:06.622 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.622 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.622 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.622 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.622 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 15, 2024-04-26T04:33:06.622 INFO:teuthology.orchestra.run.smithi012.stdout: "uuid": "9933b5aa-e091-43b1-b788-5b79695bab11", 2024-04-26T04:33:06.622 INFO:teuthology.orchestra.run.smithi012.stdout: "up": 1, 2024-04-26T04:33:06.622 INFO:teuthology.orchestra.run.smithi012.stdout: "in": 1, 2024-04-26T04:33:06.622 INFO:teuthology.orchestra.run.smithi012.stdout: "weight": 1, 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_affinity": 1, 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_begin": 0, 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: "last_clean_end": 0, 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: "up_from": 21, 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: "up_thru": 30, 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: "down_at": 0, 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: "lost_at": 0, 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addrs": { 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6802", 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6804", 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addrs": { 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6806", 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6807", 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addrs": { 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6812", 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.626 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6814", 2024-04-26T04:33:06.626 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:33:06.626 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.626 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.626 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addrs": { 2024-04-26T04:33:06.626 INFO:teuthology.orchestra.run.smithi012.stdout: "addrvec": [ 2024-04-26T04:33:06.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.626 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v2", 2024-04-26T04:33:06.626 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6809", 2024-04-26T04:33:06.626 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:33:06.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type": "v1", 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: "addr": "172.21.15.178:6811", 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: "nonce": 314165740 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: "public_addr": "172.21.15.178:6804/314165740", 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: "cluster_addr": "172.21.15.178:6807/314165740", 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_back_addr": "172.21.15.178:6814/314165740", 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: "heartbeat_front_addr": "172.21.15.178:6811/314165740", 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: "state": [ 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: "exists", 2024-04-26T04:33:06.627 INFO:teuthology.orchestra.run.smithi012.stdout: "up" 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: "osd_xinfo": [ 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 0, 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.305255+0000", 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 1, 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.420004+0000", 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 2, 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.629 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 65536, 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.396744+0000", 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 3, 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.471304+0000", 2024-04-26T04:33:06.630 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.631 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 4, 2024-04-26T04:33:06.631 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.631 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.631 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.631 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.631 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.631 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.276649+0000", 2024-04-26T04:33:06.631 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.631 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 5, 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.349822+0000", 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 6, 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.632 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.396513+0000", 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 7, 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.438157+0000", 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.633 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 8, 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.166681+0000", 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 9, 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.634 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.532725+0000", 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 10, 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.242376+0000", 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.636 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 11, 2024-04-26T04:33:06.636 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.636 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.636 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.636 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.636 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.636 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.328959+0000", 2024-04-26T04:33:06.636 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.636 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 12, 2024-04-26T04:33:06.636 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.636 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.440612+0000", 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 13, 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.485079+0000", 2024-04-26T04:33:06.637 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 14, 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.290530+0000", 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: "osd": 15, 2024-04-26T04:33:06.638 INFO:teuthology.orchestra.run.smithi012.stdout: "down_stamp": "0.000000", 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_probability": 0, 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: "laggy_interval": 0, 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: "features": 4540701547738038271, 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: "old_weight": 0, 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: "last_purged_snaps_scrub": "2024-04-26T04:32:06.232637+0000", 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: "dead_epoch": 0 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap": [], 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_items": [ 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: "pgid": "3.0", 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: "mappings": [ 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.639 INFO:teuthology.orchestra.run.smithi012.stdout: "from": 15, 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: "to": 4 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: "from": 6, 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: "to": 1 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_upmap_primaries": [], 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: "pg_temp": [], 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: "primary_temp": [], 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: "blocklist": {}, 2024-04-26T04:33:06.640 INFO:teuthology.orchestra.run.smithi012.stdout: "range_blocklist": {}, 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "erasure_code_profiles": { 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "default": { 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-failure-domain": "osd", 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "k": "2", 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "m": "1", 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin": "jerasure", 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "technique": "reed_sol_van" 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "shecprofile": { 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "c": "2", 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-device-class": "", 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-failure-domain": "osd", 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-num-failure-domains": "0", 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-osds-per-failure-domain": "0", 2024-04-26T04:33:06.641 INFO:teuthology.orchestra.run.smithi012.stdout: "crush-root": "default", 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: "k": "4", 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: "m": "3", 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: "name": "shecprofile", 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: "plugin": "shec", 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: "technique": "multiple" 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: "removed_snaps_queue": [ 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: "pool": 3, 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: "snaps": [ 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: "begin": 11, 2024-04-26T04:33:06.642 INFO:teuthology.orchestra.run.smithi012.stdout: "length": 1 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: "begin": 14, 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: "length": 2 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: "new_removed_snaps": [], 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: "new_purged_snaps": [], 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: "crush_node_flags": {}, 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: "device_class_flags": {}, 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode": { 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_enabled": false, 2024-04-26T04:33:06.643 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_bucket_count": 0, 2024-04-26T04:33:06.644 INFO:teuthology.orchestra.run.smithi012.stdout: "degraded_stretch_mode": 0, 2024-04-26T04:33:06.644 INFO:teuthology.orchestra.run.smithi012.stdout: "recovering_stretch_mode": 0, 2024-04-26T04:33:06.644 INFO:teuthology.orchestra.run.smithi012.stdout: "stretch_mode_bucket": 0 2024-04-26T04:33:06.644 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:06.644 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:06.646 INFO:tasks.rados.rados.0.smithi178.stdout:301: finishing write tid 6 to smithi17829308-38 2024-04-26T04:33:06.646 INFO:tasks.rados.rados.0.smithi178.stdout:301: oid 38 updating version 0 to 75 2024-04-26T04:33:06.646 INFO:tasks.rados.rados.0.smithi178.stdout:301: oid 38 updating version 75 to 77 2024-04-26T04:33:06.646 INFO:tasks.rados.rados.0.smithi178.stdout:301: oid 38 updating version 77 to 78 2024-04-26T04:33:06.646 INFO:tasks.rados.rados.0.smithi178.stdout:301: oid 38 version 78 is already newer than 76 2024-04-26T04:33:06.646 INFO:tasks.rados.rados.0.smithi178.stdout:301: oid 38 version 78 is already newer than 74 2024-04-26T04:33:06.647 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 38 v 78 (ObjNum 124 snap 17 seq_num 124) dirty exists 2024-04-26T04:33:06.647 INFO:tasks.rados.rados.0.smithi178.stdout:301: left oid 38 (ObjNum 124 snap 17 seq_num 124) 2024-04-26T04:33:06.647 INFO:tasks.rados.rados.0.smithi178.stdout:301: done (3 left) 2024-04-26T04:33:06.647 INFO:tasks.rados.rados.0.smithi178.stdout:304: done (2 left) 2024-04-26T04:33:06.647 INFO:tasks.rados.rados.0.smithi178.stdout:312: done (1 left) 2024-04-26T04:33:06.647 INFO:tasks.rados.rados.0.smithi178.stdout:313: delete oid 6 current snap is 18 2024-04-26T04:33:06.648 INFO:tasks.thrashosds.thrasher:j is {'epoch': 71, 'fsid': '12c27fbe-0161-48d0-88ec-42a226685437', 'created': '2024-04-26T04:31:56.451062+0000', 'modified': '2024-04-26T04:33:06.233796+0000', 'last_up_change': '2024-04-26T04:32:09.452464+0000', 'last_in_change': '2024-04-26T04:32:20.674579+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T04:32:07.426875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 15.380000114440918, 'score_stable': 15.380000114440918, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9652423858642578, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-04-26T04:32:11.318957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.019999980926514, 'score_stable': 4.019999980926514, 'optimal_score': 0.5600000023841858, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9652423858642578, 'average_primary_affinity': 0.9700000286102295, 'average_primary_affinity_weighted': 0.9700000286102295}}, {'pool': 3, 'pool_name': 'unique_pool_0', 'create_time': '2024-04-26T04:32:21.196824+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 3, 'size': 7, 'min_size': 5, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 19, 'snap_epoch': 71, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'shecprofile', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 16384, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}}], 'osds': [{'osd': 0, 'uuid': '55a4f254-2c2a-4007-9c52-b89df6fffb69', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6816', 'nonce': 1754063127}, {'type': 'v1', 'addr': '172.21.15.12:6817', 'nonce': 1754063127}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6818', 'nonce': 1754063127}, {'type': 'v1', 'addr': '172.21.15.12:6819', 'nonce': 1754063127}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6822', 'nonce': 1754063127}, {'type': 'v1', 'addr': '172.21.15.12:6823', 'nonce': 1754063127}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6820', 'nonce': 1754063127}, {'type': 'v1', 'addr': '172.21.15.12:6821', 'nonce': 1754063127}]}, 'public_addr': '172.21.15.12:6817/1754063127', 'cluster_addr': '172.21.15.12:6819/1754063127', 'heartbeat_back_addr': '172.21.15.12:6823/1754063127', 'heartbeat_front_addr': '172.21.15.12:6821/1754063127', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '32db4f97-c8ac-481c-9e80-18fc91004298', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6808', 'nonce': 3264326676}, {'type': 'v1', 'addr': '172.21.15.113:6809', 'nonce': 3264326676}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6810', 'nonce': 3264326676}, {'type': 'v1', 'addr': '172.21.15.113:6811', 'nonce': 3264326676}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6814', 'nonce': 3264326676}, {'type': 'v1', 'addr': '172.21.15.113:6817', 'nonce': 3264326676}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6812', 'nonce': 3264326676}, {'type': 'v1', 'addr': '172.21.15.113:6813', 'nonce': 3264326676}]}, 'public_addr': '172.21.15.113:6809/3264326676', 'cluster_addr': '172.21.15.113:6811/3264326676', 'heartbeat_back_addr': '172.21.15.113:6817/3264326676', 'heartbeat_front_addr': '172.21.15.113:6813/3264326676', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '2cd26466-5d59-4599-ba3f-a78b0e3b681f', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6800', 'nonce': 1454475448}, {'type': 'v1', 'addr': '172.21.15.137:6802', 'nonce': 1454475448}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6804', 'nonce': 1454475448}, {'type': 'v1', 'addr': '172.21.15.137:6806', 'nonce': 1454475448}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6812', 'nonce': 1454475448}, {'type': 'v1', 'addr': '172.21.15.137:6814', 'nonce': 1454475448}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6808', 'nonce': 1454475448}, {'type': 'v1', 'addr': '172.21.15.137:6810', 'nonce': 1454475448}]}, 'public_addr': '172.21.15.137:6802/1454475448', 'cluster_addr': '172.21.15.137:6806/1454475448', 'heartbeat_back_addr': '172.21.15.137:6814/1454475448', 'heartbeat_front_addr': '172.21.15.137:6810/1454475448', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '0945017d-9525-49d6-aa67-6635acd94c3e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6803', 'nonce': 2898698985}, {'type': 'v1', 'addr': '172.21.15.178:6805', 'nonce': 2898698985}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6808', 'nonce': 2898698985}, {'type': 'v1', 'addr': '172.21.15.178:6810', 'nonce': 2898698985}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6816', 'nonce': 2898698985}, {'type': 'v1', 'addr': '172.21.15.178:6817', 'nonce': 2898698985}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6813', 'nonce': 2898698985}, {'type': 'v1', 'addr': '172.21.15.178:6815', 'nonce': 2898698985}]}, 'public_addr': '172.21.15.178:6805/2898698985', 'cluster_addr': '172.21.15.178:6810/2898698985', 'heartbeat_back_addr': '172.21.15.178:6817/2898698985', 'heartbeat_front_addr': '172.21.15.178:6815/2898698985', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '38dc68bf-0a89-4900-9d35-24c70368447a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6808', 'nonce': 3209840823}, {'type': 'v1', 'addr': '172.21.15.12:6809', 'nonce': 3209840823}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6810', 'nonce': 3209840823}, {'type': 'v1', 'addr': '172.21.15.12:6811', 'nonce': 3209840823}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6814', 'nonce': 3209840823}, {'type': 'v1', 'addr': '172.21.15.12:6815', 'nonce': 3209840823}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6812', 'nonce': 3209840823}, {'type': 'v1', 'addr': '172.21.15.12:6813', 'nonce': 3209840823}]}, 'public_addr': '172.21.15.12:6809/3209840823', 'cluster_addr': '172.21.15.12:6811/3209840823', 'heartbeat_back_addr': '172.21.15.12:6815/3209840823', 'heartbeat_front_addr': '172.21.15.12:6813/3209840823', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'e91cc8a1-57e1-4760-a329-8a60aa752a95', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6800', 'nonce': 1475637773}, {'type': 'v1', 'addr': '172.21.15.113:6801', 'nonce': 1475637773}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6802', 'nonce': 1475637773}, {'type': 'v1', 'addr': '172.21.15.113:6803', 'nonce': 1475637773}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6806', 'nonce': 1475637773}, {'type': 'v1', 'addr': '172.21.15.113:6807', 'nonce': 1475637773}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6804', 'nonce': 1475637773}, {'type': 'v1', 'addr': '172.21.15.113:6805', 'nonce': 1475637773}]}, 'public_addr': '172.21.15.113:6801/1475637773', 'cluster_addr': '172.21.15.113:6803/1475637773', 'heartbeat_back_addr': '172.21.15.113:6807/1475637773', 'heartbeat_front_addr': '172.21.15.113:6805/1475637773', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'a1322130-3391-432d-a1ee-f83297885029', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 25, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6816', 'nonce': 771338783}, {'type': 'v1', 'addr': '172.21.15.137:6817', 'nonce': 771338783}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6818', 'nonce': 771338783}, {'type': 'v1', 'addr': '172.21.15.137:6819', 'nonce': 771338783}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6822', 'nonce': 771338783}, {'type': 'v1', 'addr': '172.21.15.137:6823', 'nonce': 771338783}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6820', 'nonce': 771338783}, {'type': 'v1', 'addr': '172.21.15.137:6821', 'nonce': 771338783}]}, 'public_addr': '172.21.15.137:6817/771338783', 'cluster_addr': '172.21.15.137:6819/771338783', 'heartbeat_back_addr': '172.21.15.137:6823/771338783', 'heartbeat_front_addr': '172.21.15.137:6821/771338783', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '02ee4ca2-9d48-4aff-9641-1823760c7a3b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6826', 'nonce': 1017152077}, {'type': 'v1', 'addr': '172.21.15.178:6827', 'nonce': 1017152077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6828', 'nonce': 1017152077}, {'type': 'v1', 'addr': '172.21.15.178:6829', 'nonce': 1017152077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6832', 'nonce': 1017152077}, {'type': 'v1', 'addr': '172.21.15.178:6833', 'nonce': 1017152077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6830', 'nonce': 1017152077}, {'type': 'v1', 'addr': '172.21.15.178:6831', 'nonce': 1017152077}]}, 'public_addr': '172.21.15.178:6827/1017152077', 'cluster_addr': '172.21.15.178:6829/1017152077', 'heartbeat_back_addr': '172.21.15.178:6833/1017152077', 'heartbeat_front_addr': '172.21.15.178:6831/1017152077', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': 'ea3d5f62-3d68-4824-8556-da8db36553fb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6800', 'nonce': 2153129138}, {'type': 'v1', 'addr': '172.21.15.12:6801', 'nonce': 2153129138}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6802', 'nonce': 2153129138}, {'type': 'v1', 'addr': '172.21.15.12:6803', 'nonce': 2153129138}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6806', 'nonce': 2153129138}, {'type': 'v1', 'addr': '172.21.15.12:6807', 'nonce': 2153129138}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6804', 'nonce': 2153129138}, {'type': 'v1', 'addr': '172.21.15.12:6805', 'nonce': 2153129138}]}, 'public_addr': '172.21.15.12:6801/2153129138', 'cluster_addr': '172.21.15.12:6803/2153129138', 'heartbeat_back_addr': '172.21.15.12:6807/2153129138', 'heartbeat_front_addr': '172.21.15.12:6805/2153129138', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': 'e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6824', 'nonce': 359931551}, {'type': 'v1', 'addr': '172.21.15.113:6825', 'nonce': 359931551}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6826', 'nonce': 359931551}, {'type': 'v1', 'addr': '172.21.15.113:6827', 'nonce': 359931551}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6830', 'nonce': 359931551}, {'type': 'v1', 'addr': '172.21.15.113:6831', 'nonce': 359931551}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6828', 'nonce': 359931551}, {'type': 'v1', 'addr': '172.21.15.113:6829', 'nonce': 359931551}]}, 'public_addr': '172.21.15.113:6825/359931551', 'cluster_addr': '172.21.15.113:6827/359931551', 'heartbeat_back_addr': '172.21.15.113:6831/359931551', 'heartbeat_front_addr': '172.21.15.113:6829/359931551', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '8c4f8df5-465a-486e-9413-1de14e33c119', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6801', 'nonce': 1533480861}, {'type': 'v1', 'addr': '172.21.15.137:6803', 'nonce': 1533480861}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6805', 'nonce': 1533480861}, {'type': 'v1', 'addr': '172.21.15.137:6807', 'nonce': 1533480861}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6813', 'nonce': 1533480861}, {'type': 'v1', 'addr': '172.21.15.137:6815', 'nonce': 1533480861}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6809', 'nonce': 1533480861}, {'type': 'v1', 'addr': '172.21.15.137:6811', 'nonce': 1533480861}]}, 'public_addr': '172.21.15.137:6803/1533480861', 'cluster_addr': '172.21.15.137:6807/1533480861', 'heartbeat_back_addr': '172.21.15.137:6815/1533480861', 'heartbeat_front_addr': '172.21.15.137:6811/1533480861', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '6fe54c2d-0b95-4235-8c22-51230cfaa591', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6818', 'nonce': 1683682957}, {'type': 'v1', 'addr': '172.21.15.178:6819', 'nonce': 1683682957}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6820', 'nonce': 1683682957}, {'type': 'v1', 'addr': '172.21.15.178:6821', 'nonce': 1683682957}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6824', 'nonce': 1683682957}, {'type': 'v1', 'addr': '172.21.15.178:6825', 'nonce': 1683682957}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6822', 'nonce': 1683682957}, {'type': 'v1', 'addr': '172.21.15.178:6823', 'nonce': 1683682957}]}, 'public_addr': '172.21.15.178:6819/1683682957', 'cluster_addr': '172.21.15.178:6821/1683682957', 'heartbeat_back_addr': '172.21.15.178:6825/1683682957', 'heartbeat_front_addr': '172.21.15.178:6823/1683682957', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': '5ddd0f05-fb78-43bb-95ec-3fc7985603b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6824', 'nonce': 2509311840}, {'type': 'v1', 'addr': '172.21.15.12:6825', 'nonce': 2509311840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6826', 'nonce': 2509311840}, {'type': 'v1', 'addr': '172.21.15.12:6827', 'nonce': 2509311840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6830', 'nonce': 2509311840}, {'type': 'v1', 'addr': '172.21.15.12:6831', 'nonce': 2509311840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.12:6828', 'nonce': 2509311840}, {'type': 'v1', 'addr': '172.21.15.12:6829', 'nonce': 2509311840}]}, 'public_addr': '172.21.15.12:6825/2509311840', 'cluster_addr': '172.21.15.12:6827/2509311840', 'heartbeat_back_addr': '172.21.15.12:6831/2509311840', 'heartbeat_front_addr': '172.21.15.12:6829/2509311840', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '37eb42c1-f29e-49f5-b8f6-035178038e5d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.443878173828125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6815', 'nonce': 3716239663}, {'type': 'v1', 'addr': '172.21.15.113:6816', 'nonce': 3716239663}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6818', 'nonce': 3716239663}, {'type': 'v1', 'addr': '172.21.15.113:6819', 'nonce': 3716239663}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6822', 'nonce': 3716239663}, {'type': 'v1', 'addr': '172.21.15.113:6823', 'nonce': 3716239663}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6820', 'nonce': 3716239663}, {'type': 'v1', 'addr': '172.21.15.113:6821', 'nonce': 3716239663}]}, 'public_addr': '172.21.15.113:6816/3716239663', 'cluster_addr': '172.21.15.113:6819/3716239663', 'heartbeat_back_addr': '172.21.15.113:6823/3716239663', 'heartbeat_front_addr': '172.21.15.113:6821/3716239663', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': 'b2267367-a857-4a5c-a8d1-6a9b23c748f6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6824', 'nonce': 1454956132}, {'type': 'v1', 'addr': '172.21.15.137:6825', 'nonce': 1454956132}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6826', 'nonce': 1454956132}, {'type': 'v1', 'addr': '172.21.15.137:6827', 'nonce': 1454956132}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6830', 'nonce': 1454956132}, {'type': 'v1', 'addr': '172.21.15.137:6831', 'nonce': 1454956132}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6828', 'nonce': 1454956132}, {'type': 'v1', 'addr': '172.21.15.137:6829', 'nonce': 1454956132}]}, 'public_addr': '172.21.15.137:6825/1454956132', 'cluster_addr': '172.21.15.137:6827/1454956132', 'heartbeat_back_addr': '172.21.15.137:6831/1454956132', 'heartbeat_front_addr': '172.21.15.137:6829/1454956132', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '9933b5aa-e091-43b1-b788-5b79695bab11', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6802', 'nonce': 314165740}, {'type': 'v1', 'addr': '172.21.15.178:6804', 'nonce': 314165740}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6806', 'nonce': 314165740}, {'type': 'v1', 'addr': '172.21.15.178:6807', 'nonce': 314165740}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6812', 'nonce': 314165740}, {'type': 'v1', 'addr': '172.21.15.178:6814', 'nonce': 314165740}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.178:6809', 'nonce': 314165740}, {'type': 'v1', 'addr': '172.21.15.178:6811', 'nonce': 314165740}]}, 'public_addr': '172.21.15.178:6804/314165740', 'cluster_addr': '172.21.15.178:6807/314165740', 'heartbeat_back_addr': '172.21.15.178:6814/314165740', 'heartbeat_front_addr': '172.21.15.178:6811/314165740', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.305255+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.420004+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.396744+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.471304+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.276649+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.349822+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.396513+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.438157+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.166681+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.532725+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.242376+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.328959+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.440612+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.485079+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.290530+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-26T04:32:06.232637+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.0', 'mappings': [{'from': 15, 'to': 4}, {'from': 6, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}, 'shecprofile': {'c': '2', 'crush-device-class': '', 'crush-failure-domain': 'osd', 'crush-num-failure-domains': '0', 'crush-osds-per-failure-domain': '0', 'crush-root': 'default', 'k': '4', 'm': '3', 'name': 'shecprofile', 'plugin': 'shec', 'technique': 'multiple'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 11, 'length': 1}, {'begin': 14, 'length': 2}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-26T04:33:06.648 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:33:06.649 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:06.645+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:06.746 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:06.745+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:06.751 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T04:33:06.771 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:06.771 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:06.776 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 46 v 80 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-26T04:33:06.786 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T04:33:06.847 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:06.844+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:06.879 INFO:tasks.rados.rados.0.smithi178.stdout:303: done (1 left) 2024-04-26T04:33:06.879 INFO:tasks.rados.rados.0.smithi178.stdout:313: done (0 left) 2024-04-26T04:33:06.879 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 42 current snap is 18 2024-04-26T04:33:06.879 INFO:tasks.rados.rados.0.smithi178.stdout:314: seq_num 128 ranges {638976=606208,1245184=507904,1753088=573440,2326528=475136} 2024-04-26T04:33:06.893 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:33:06.893 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:33:06.899 INFO:tasks.rados.rados.0.smithi178.stdout:314: writing smithi17829308-42 from 638976 to 1245184 tid 1 2024-04-26T04:33:06.908 INFO:tasks.rados.rados.0.smithi178.stdout:314: writing smithi17829308-42 from 1245184 to 1753088 tid 2 2024-04-26T04:33:06.908 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:06.909 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:06.916 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:33:06.916 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:33:06.917 INFO:tasks.rados.rados.0.smithi178.stdout:314: writing smithi17829308-42 from 1753088 to 2326528 tid 3 2024-04-26T04:33:06.922 INFO:tasks.rados.rados.0.smithi178.stdout:314: writing smithi17829308-42 from 2326528 to 2801664 tid 4 2024-04-26T04:33:06.923 INFO:tasks.rados.rados.0.smithi178.stdout:315: snap_create 2024-04-26T04:33:06.924 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T04:33:06.937 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:33:06.937 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:33:06.948 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:06.945+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:06.955 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:33:06.955 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:33:06.972 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:33:06.972 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:33:06.992 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:33:06.992 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:33:07.008 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:33:07.009 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:33:07.013 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:33:07.013 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:33:07.027 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-04-26T04:33:06.238926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":78578208,"num_objects":73,"num_object_clones":25,"num_object_copies":501,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":6,"num_read":547,"num_read_kb":164129,"num_write":774,"num_write_kb":99844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":839,"ondisk_log_size":839,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":15,"num_per_pool_osds":15,"num_per_pool_omap_osds":15,"kb":1415577600,"kb_used":604924,"kb_used_data":149652,"kb_used_omap":23,"kb_used_meta":455016,"kb_avail":1414972676,"statfs":{"total":1449551462400,"available":1448932020224,"internally_reserved":0,"allocated":153243648,"data_stored":182060989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23926,"internal_metadata":465937034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,6,6,0,2,2,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10551296,"num_objects":-5,"num_object_clones":-2,"num_object_copies":-35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":1,"num_read":46,"num_read_kb":26868,"num_write":75,"num_write_kb":14912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.028020"},"pg_stats":[{"pgid":"3.f","version":"68'68","reported_seq":270,"reported_epoch":69,"state":"active+clean+snaptrim","last_fresh":"2024-04-26T04:33:04.340876+0000","last_change":"2024-04-26T04:33:04.340936+0000","last_active":"2024-04-26T04:33:04.340876+0000","last_peered":"2024-04-26T04:33:04.340876+0000","last_clean":"2024-04-26T04:33:04.340876+0000","last_became_active":"2024-04-26T04:32:23.378967+0000","last_became_peered":"2024-04-26T04:32:23.378967+0000","last_unstale":"2024-04-26T04:33:04.340876+0000","last_undegraded":"2024-04-26T04:33:04.340876+0000","last_fullsized":"2024-04-26T04:33:04.340876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:32.144658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.0939624210000001,"stat_sum":{"num_bytes":2031616,"num_objects":7,"num_object_clones":2,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":50,"num_read_kb":7338,"num_write":82,"num_write_kb":6608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,14,13,8,10,0],"acting":[11,1,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"b","length":"1"},{"start":"f","length":"1"}]},{"pgid":"3.e","version":"69'56","reported_seq":241,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T04:33:05.442753+0000","last_change":"2024-04-26T04:33:04.952481+0000","last_active":"2024-04-26T04:33:05.442753+0000","last_peered":"2024-04-26T04:33:05.442753+0000","last_clean":"2024-04-26T04:33:05.442753+0000","last_became_active":"2024-04-26T04:32:23.387938+0000","last_became_peered":"2024-04-26T04:32:23.387938+0000","last_unstale":"2024-04-26T04:33:05.442753+0000","last_undegraded":"2024-04-26T04:33:05.442753+0000","last_fullsized":"2024-04-26T04:33:05.442753+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:34.971321+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.38581605600000002,"stat_sum":{"num_bytes":7913472,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":18,"num_read_kb":9273,"num_write":31,"num_write_kb":8032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,9,1,11,8,6,12],"acting":[13,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"3.d","version":"69'56","reported_seq":245,"reported_epoch":69,"state":"active+clean+snaptrim","last_fresh":"2024-04-26T04:33:04.416953+0000","last_change":"2024-04-26T04:33:04.416953+0000","last_active":"2024-04-26T04:33:04.416953+0000","last_peered":"2024-04-26T04:33:04.416953+0000","last_clean":"2024-04-26T04:33:04.416953+0000","last_became_active":"2024-04-26T04:32:23.392585+0000","last_became_peered":"2024-04-26T04:32:23.392585+0000","last_unstale":"2024-04-26T04:33:04.416953+0000","last_undegraded":"2024-04-26T04:33:04.416953+0000","last_fullsized":"2024-04-26T04:33:04.416953+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:49.423678+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.82016541899999995,"stat_sum":{"num_bytes":3342336,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":9755,"num_write":67,"num_write_kb":6352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"b","length":"1"},{"start":"f","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":177,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-26T04:33:04.263769+0000","last_change":"2024-04-26T04:33:04.264585+0000","last_active":"2024-04-26T04:33:04.263769+0000","last_peered":"2024-04-26T04:33:04.263769+0000","last_clean":"2024-04-26T04:33:04.263769+0000","last_became_active":"2024-04-26T04:32:23.577814+0000","last_became_peered":"2024-04-26T04:32:23.577814+0000","last_unstale":"2024-04-26T04:33:04.263769+0000","last_undegraded":"2024-04-26T04:33:04.263769+0000","last_fullsized":"2024-04-26T04:33:04.263769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:34.061650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012010549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"3.b","version":"66'43","reported_seq":233,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-26T04:33:02.396143+0000","last_change":"2024-04-26T04:33:00.274159+0000","last_active":"2024-04-26T04:33:02.396143+0000","last_peered":"2024-04-26T04:33:02.396143+0000","last_clean":"2024-04-26T04:33:02.396143+0000","last_became_active":"2024-04-26T04:32:23.220443+0000","last_became_peered":"2024-04-26T04:32:23.220443+0000","last_unstale":"2024-04-26T04:33:02.396143+0000","last_undegraded":"2024-04-26T04:33:02.396143+0000","last_fullsized":"2024-04-26T04:33:02.396143+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:33.492668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0064591780000000003,"stat_sum":{"num_bytes":7192576,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":48,"num_read_kb":23199,"num_write":28,"num_write_kb":8912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"f","length":"1"}]},{"pgid":"3.a","version":"68'41","reported_seq":225,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-26T04:33:04.229090+0000","last_change":"2024-04-26T04:33:04.229666+0000","last_active":"2024-04-26T04:33:04.229090+0000","last_peered":"2024-04-26T04:33:04.229090+0000","last_clean":"2024-04-26T04:33:04.229090+0000","last_became_active":"2024-04-26T04:32:23.310795+0000","last_became_peered":"2024-04-26T04:32:23.310795+0000","last_unstale":"2024-04-26T04:33:04.229090+0000","last_undegraded":"2024-04-26T04:33:04.229090+0000","last_fullsized":"2024-04-26T04:33:04.229090+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:23.967667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093880899999999998,"stat_sum":{"num_bytes":3309568,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":1350,"num_write":18,"num_write_kb":5040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"3.9","version":"69'12","reported_seq":202,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T04:33:05.338373+0000","last_change":"2024-04-26T04:33:04.690826+0000","last_active":"2024-04-26T04:33:05.338373+0000","last_peered":"2024-04-26T04:33:05.338373+0000","last_clean":"2024-04-26T04:33:05.338373+0000","last_became_active":"2024-04-26T04:32:23.476584+0000","last_became_peered":"2024-04-26T04:32:23.476584+0000","last_unstale":"2024-04-26T04:33:05.338373+0000","last_undegraded":"2024-04-26T04:33:05.338373+0000","last_fullsized":"2024-04-26T04:33:05.338373+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:48.647681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20485212699999999,"stat_sum":{"num_bytes":458752,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":15,"num_read_kb":4132,"num_write":23,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"3.8","version":"69'95","reported_seq":292,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T04:33:05.395829+0000","last_change":"2024-04-26T04:33:05.295712+0000","last_active":"2024-04-26T04:33:05.395829+0000","last_peered":"2024-04-26T04:33:05.395829+0000","last_clean":"2024-04-26T04:33:05.395829+0000","last_became_active":"2024-04-26T04:32:23.487565+0000","last_became_peered":"2024-04-26T04:32:23.487565+0000","last_unstale":"2024-04-26T04:33:05.395829+0000","last_undegraded":"2024-04-26T04:33:05.395829+0000","last_fullsized":"2024-04-26T04:33:05.395829+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:39.460031+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.833921735,"stat_sum":{"num_bytes":10764288,"num_objects":7,"num_object_clones":1,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":45,"num_read_kb":15761,"num_write":81,"num_write_kb":11728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,12,1,14,15,6],"acting":[5,0,12,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"1.0","version":"32'157","reported_seq":259,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-26T04:33:04.228299+0000","last_change":"2024-04-26T04:32:56.378977+0000","last_active":"2024-04-26T04:33:04.228299+0000","last_peered":"2024-04-26T04:33:04.228299+0000","last_clean":"2024-04-26T04:33:04.228299+0000","last_became_active":"2024-04-26T04:32:56.378173+0000","last_became_peered":"2024-04-26T04:32:56.378173+0000","last_unstale":"2024-04-26T04:33:04.228299+0000","last_undegraded":"2024-04-26T04:33:04.228299+0000","last_fullsized":"2024-04-26T04:33:04.228299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:32:08.365921+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:25.422212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,13],"acting":[3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":98,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T04:33:05.339614+0000","last_change":"2024-04-26T04:32:12.505010+0000","last_active":"2024-04-26T04:33:05.339614+0000","last_peered":"2024-04-26T04:33:05.339614+0000","last_clean":"2024-04-26T04:33:05.339614+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:33:05.339614+0000","last_undegraded":"2024-04-26T04:33:05.339614+0000","last_fullsized":"2024-04-26T04:33:05.339614+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.374592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"44'12","reported_seq":121,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-26T04:33:02.406225+0000","last_change":"2024-04-26T04:33:00.436188+0000","last_active":"2024-04-26T04:33:02.406225+0000","last_peered":"2024-04-26T04:33:02.406225+0000","last_clean":"2024-04-26T04:33:02.406225+0000","last_became_active":"2024-04-26T04:32:56.531093+0000","last_became_peered":"2024-04-26T04:32:56.531093+0000","last_unstale":"2024-04-26T04:33:02.406225+0000","last_undegraded":"2024-04-26T04:33:02.406225+0000","last_fullsized":"2024-04-26T04:33:02.406225+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:41.204688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.101149041,"stat_sum":{"num_bytes":360448,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":17,"num_read_kb":3174,"num_write":8,"num_write_kb":1584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10,3,6,15,1,11],"acting":[13,10,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"f","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":95,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-26T04:33:04.328464+0000","last_change":"2024-04-26T04:32:12.603501+0000","last_active":"2024-04-26T04:33:04.328464+0000","last_peered":"2024-04-26T04:33:04.328464+0000","last_clean":"2024-04-26T04:33:04.328464+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:33:04.328464+0000","last_undegraded":"2024-04-26T04:33:04.328464+0000","last_fullsized":"2024-04-26T04:33:04.328464+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.945207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"65'76","reported_seq":254,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-26T04:33:02.213488+0000","last_change":"2024-04-26T04:33:00.268091+0000","last_active":"2024-04-26T04:33:02.213488+0000","last_peered":"2024-04-26T04:33:02.213488+0000","last_clean":"2024-04-26T04:33:02.213488+0000","last_became_active":"2024-04-26T04:32:23.399115+0000","last_became_peered":"2024-04-26T04:32:23.399115+0000","last_unstale":"2024-04-26T04:33:02.213488+0000","last_undegraded":"2024-04-26T04:33:02.213488+0000","last_fullsized":"2024-04-26T04:33:02.213488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:50.840092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067132900000000002,"stat_sum":{"num_bytes":12976128,"num_objects":8,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":37,"num_read_kb":29442,"num_write":53,"num_write_kb":15456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,12,13],"acting":[4,11,9,5,10,12,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"f","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T04:33:05.339359+0000","last_change":"2024-04-26T04:32:56.277333+0000","last_active":"2024-04-26T04:33:05.339359+0000","last_peered":"2024-04-26T04:33:05.339359+0000","last_clean":"2024-04-26T04:33:05.339359+0000","last_became_active":"2024-04-26T04:32:56.276685+0000","last_became_peered":"2024-04-26T04:32:56.276685+0000","last_unstale":"2024-04-26T04:33:05.339359+0000","last_undegraded":"2024-04-26T04:33:05.339359+0000","last_fullsized":"2024-04-26T04:33:05.339359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.299653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,13],"acting":[5,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"69'17","reported_seq":204,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T04:33:05.338130+0000","last_change":"2024-04-26T04:33:04.892513+0000","last_active":"2024-04-26T04:33:05.338130+0000","last_peered":"2024-04-26T04:33:05.338130+0000","last_clean":"2024-04-26T04:33:05.338130+0000","last_became_active":"2024-04-26T04:32:23.386975+0000","last_became_peered":"2024-04-26T04:32:23.386975+0000","last_unstale":"2024-04-26T04:33:05.338130+0000","last_undegraded":"2024-04-26T04:33:05.338130+0000","last_fullsized":"2024-04-26T04:33:05.338130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:46.729754+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.40647535099999998,"stat_sum":{"num_bytes":1654784,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":4851,"num_write":10,"num_write_kb":1744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,11,15,1,5,6,4],"acting":[14,11,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"2.2","version":"0'0","reported_seq":95,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-26T04:33:04.228611+0000","last_change":"2024-04-26T04:32:12.798464+0000","last_active":"2024-04-26T04:33:04.228611+0000","last_peered":"2024-04-26T04:33:04.228611+0000","last_clean":"2024-04-26T04:33:04.228611+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:33:04.228611+0000","last_undegraded":"2024-04-26T04:33:04.228611+0000","last_fullsized":"2024-04-26T04:33:04.228611+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.684262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":180,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T04:33:05.342364+0000","last_change":"2024-04-26T04:33:04.366572+0000","last_active":"2024-04-26T04:33:05.342364+0000","last_peered":"2024-04-26T04:33:05.342364+0000","last_clean":"2024-04-26T04:33:05.342364+0000","last_became_active":"2024-04-26T04:32:23.387136+0000","last_became_peered":"2024-04-26T04:32:23.387136+0000","last_unstale":"2024-04-26T04:33:05.342364+0000","last_undegraded":"2024-04-26T04:33:05.342364+0000","last_fullsized":"2024-04-26T04:33:05.342364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:29.368041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012675239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,6,5,3],"acting":[13,7,4,15,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"3.5","version":"66'54","reported_seq":234,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-26T04:33:02.596989+0000","last_change":"2024-04-26T04:33:00.281259+0000","last_active":"2024-04-26T04:33:02.596989+0000","last_peered":"2024-04-26T04:33:02.596989+0000","last_clean":"2024-04-26T04:33:02.596989+0000","last_became_active":"2024-04-26T04:32:23.295127+0000","last_became_peered":"2024-04-26T04:32:23.295127+0000","last_unstale":"2024-04-26T04:33:02.596989+0000","last_undegraded":"2024-04-26T04:33:02.596989+0000","last_fullsized":"2024-04-26T04:33:02.596989+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:36.711388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058014700000000004,"stat_sum":{"num_bytes":4964352,"num_objects":9,"num_object_clones":4,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":32,"num_read_kb":12588,"num_write":49,"num_write_kb":5712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"f","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":90,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-26T04:33:02.213618+0000","last_change":"2024-04-26T04:32:13.108823+0000","last_active":"2024-04-26T04:33:02.213618+0000","last_peered":"2024-04-26T04:33:02.213618+0000","last_clean":"2024-04-26T04:33:02.213618+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:33:02.213618+0000","last_undegraded":"2024-04-26T04:33:02.213618+0000","last_fullsized":"2024-04-26T04:33:02.213618+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"67'65","reported_seq":242,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-26T04:33:02.309155+0000","last_change":"2024-04-26T04:33:00.406527+0000","last_active":"2024-04-26T04:33:02.309155+0000","last_peered":"2024-04-26T04:33:02.309155+0000","last_clean":"2024-04-26T04:33:02.309155+0000","last_became_active":"2024-04-26T04:32:23.290645+0000","last_became_peered":"2024-04-26T04:32:23.290645+0000","last_unstale":"2024-04-26T04:33:02.309155+0000","last_undegraded":"2024-04-26T04:33:02.309155+0000","last_fullsized":"2024-04-26T04:33:02.309155+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:48.005545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.103040373,"stat_sum":{"num_bytes":9420800,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":34,"num_read_kb":12396,"num_write":34,"num_write_kb":10080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,4,1,0,14,13],"acting":[8,9,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"f","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":97,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-04-26T04:33:05.298244+0000","last_change":"2024-04-26T04:32:12.504400+0000","last_active":"2024-04-26T04:33:05.298244+0000","last_peered":"2024-04-26T04:33:05.298244+0000","last_clean":"2024-04-26T04:33:05.298244+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:33:05.298244+0000","last_undegraded":"2024-04-26T04:33:05.298244+0000","last_fullsized":"2024-04-26T04:33:05.298244+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.381954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"60'48","reported_seq":236,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-26T04:33:02.806197+0000","last_change":"2024-04-26T04:33:00.268766+0000","last_active":"2024-04-26T04:33:02.806197+0000","last_peered":"2024-04-26T04:33:02.806197+0000","last_clean":"2024-04-26T04:33:02.806197+0000","last_became_active":"2024-04-26T04:32:23.477983+0000","last_became_peered":"2024-04-26T04:32:23.477983+0000","last_unstale":"2024-04-26T04:33:02.806197+0000","last_undegraded":"2024-04-26T04:33:02.806197+0000","last_fullsized":"2024-04-26T04:33:02.806197+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:36.288619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083490400000000003,"stat_sum":{"num_bytes":7634944,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":36,"num_read_kb":20614,"num_write":26,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"f","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":90,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-26T04:33:02.308993+0000","last_change":"2024-04-26T04:32:12.697838+0000","last_active":"2024-04-26T04:33:02.308993+0000","last_peered":"2024-04-26T04:33:02.308993+0000","last_clean":"2024-04-26T04:33:02.308993+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:33:02.308993+0000","last_undegraded":"2024-04-26T04:33:02.308993+0000","last_fullsized":"2024-04-26T04:33:02.308993+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.961816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"65'39","reported_seq":213,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-26T04:33:02.295510+0000","last_change":"2024-04-26T04:33:00.507316+0000","last_active":"2024-04-26T04:33:02.295510+0000","last_peered":"2024-04-26T04:33:02.295510+0000","last_clean":"2024-04-26T04:33:02.295510+0000","last_became_active":"2024-04-26T04:32:23.383105+0000","last_became_peered":"2024-04-26T04:32:23.383105+0000","last_unstale":"2024-04-26T04:33:02.295510+0000","last_undegraded":"2024-04-26T04:33:02.295510+0000","last_fullsized":"2024-04-26T04:33:02.295510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.551820+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.23376486199999999,"stat_sum":{"num_bytes":5963776,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":20,"num_read_kb":10138,"num_write":61,"num_write_kb":5824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"f","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":91,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-04-26T04:33:02.398823+0000","last_change":"2024-04-26T04:32:12.803651+0000","last_active":"2024-04-26T04:33:02.398823+0000","last_peered":"2024-04-26T04:33:02.398823+0000","last_clean":"2024-04-26T04:33:02.398823+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:33:02.398823+0000","last_undegraded":"2024-04-26T04:33:02.398823+0000","last_fullsized":"2024-04-26T04:33:02.398823+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:26.614652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":77987840,"num_objects":71,"num_object_clones":25,"num_object_copies":497,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":6,"num_read":411,"num_read_kb":164011,"num_write":571,"num_write_kb":96384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":143282176,"data_stored":178049024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":682,"ondisk_log_size":682,"up":112,"acting":112,"num_store_stats":15},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1536000,"data_stored":1525344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247823,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44636,"kb_used_data":13452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327204,"statfs":{"total":96636764160,"available":96591056896,"internally_reserved":0,"allocated":13774848,"data_stored":15763080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247824,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39644,"kb_used_data":8460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332196,"statfs":{"total":96636764160,"available":96596168704,"internally_reserved":0,"allocated":8663040,"data_stored":8459912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1,0,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247823,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44148,"kb_used_data":12960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327692,"statfs":{"total":96636764160,"available":96591556608,"internally_reserved":0,"allocated":13271040,"data_stored":15372574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313230,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44104,"kb_used_data":12928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327736,"statfs":{"total":96636764160,"available":96591601664,"internally_reserved":0,"allocated":13238272,"data_stored":15887909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313230,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38996,"kb_used_data":11968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94332844,"statfs":{"total":96636764160,"available":96596832256,"internally_reserved":0,"allocated":12255232,"data_stored":15742252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,1,0,1,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313230,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36648,"kb_used_data":9620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335192,"statfs":{"total":96636764160,"available":96599236608,"internally_reserved":0,"allocated":9850880,"data_stored":11826476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313230,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33472,"kb_used_data":6452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338368,"statfs":{"total":96636764160,"available":96602488832,"internally_reserved":0,"allocated":6606848,"data_stored":9020716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,3,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313230,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33828,"kb_used_data":6812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338012,"statfs":{"total":96636764160,"available":96602124288,"internally_reserved":0,"allocated":6975488,"data_stored":9307436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":98784247824,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280526,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30824,"kb_used_data":3804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341016,"statfs":{"total":96636764160,"available":96605200384,"internally_reserved":0,"allocated":3895296,"data_stored":3698682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280526,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45032,"kb_used_data":9688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326808,"statfs":{"total":96636764160,"available":96590651392,"internally_reserved":0,"allocated":9920512,"data_stored":12459482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247823,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44396,"kb_used_data":13208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327444,"statfs":{"total":96636764160,"available":96591302656,"internally_reserved":0,"allocated":13524992,"data_stored":14610174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247824,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39276,"kb_used_data":8092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332564,"statfs":{"total":96636764160,"available":96596545536,"internally_reserved":0,"allocated":8286208,"data_stored":9995912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247822,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42400,"kb_used_data":11216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329440,"statfs":{"total":96636764160,"available":96593346560,"internally_reserved":0,"allocated":11485184,"data_stored":13923976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280526,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43852,"kb_used_data":8512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94327988,"statfs":{"total":96636764160,"available":96591859712,"internally_reserved":0,"allocated":8716288,"data_stored":10141146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247824,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43668,"kb_used_data":12480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328172,"statfs":{"total":96636764160,"available":96592048128,"internally_reserved":0,"allocated":12779520,"data_stored":15851262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14307328,"data_stored":17850368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13582336,"data_stored":16220160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1867776,"data_stored":2093056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12857344,"data_stored":16773120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7188480,"data_stored":8208384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10321920,"data_stored":11112448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7155712,"data_stored":8073216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":10227712,"data_stored":12517376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":13336576,"data_stored":16855040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":5709824,"data_stored":8536064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":7901184,"data_stored":10858496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":12185600,"data_stored":15687680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":12013568,"data_stored":15654912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":8077312,"data_stored":8314880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":6549504,"data_stored":9293824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:33:07.029 INFO:tasks.thrashosds.thrasher:Setting 3.a to [3, 1, 7, 13, 6, 9, 8] 2024-04-26T04:33:07.029 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.a', '3', '1', '7', '13', '6', '9', '8'] 2024-04-26T04:33:07.029 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.a 3 1 7 13 6 9 8 2024-04-26T04:33:07.033 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:33:07.033 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:33:07.046 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:07.046 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:07.049 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:07.049+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:07.051 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:33:07.051 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:33:07.060 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T04:33:07.072 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:33:07.072 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:33:07.093 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:33:07.094 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:33:07.111 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:33:07.111 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:33:07.112 INFO:tasks.rados.rados.0.smithi178.stdout:314: finishing write tid 1 to smithi17829308-42 2024-04-26T04:33:07.129 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:33:07.129 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:33:07.150 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:33:07.150 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:33:07.150 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:07.146+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:07.171 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:33:07.171 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:33:07.184 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:07.184 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:07.184 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:33:07.184 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:33:07.184 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:07.192 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:33:07.192 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:33:07.199 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T04:33:07.212 INFO:tasks.rados.rados.0.smithi178.stdout:314: finishing write tid 2 to smithi17829308-42 2024-04-26T04:33:07.255 INFO:tasks.rados.rados.0.smithi178.stdout:315: done (1 left) 2024-04-26T04:33:07.255 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 24 current snap is 19 2024-04-26T04:33:07.255 INFO:tasks.rados.rados.0.smithi178.stdout:316: seq_num 129 ranges {393216=622592,1015808=655360,1671168=688128,2359296=212992} 2024-04-26T04:33:07.255 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:07.254+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:07.262 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-26T04:33:07.263 INFO:tasks.rados.rados.0.smithi178.stdout:316: writing smithi17829308-24 from 393216 to 1015808 tid 1 2024-04-26T04:33:07.268 INFO:tasks.rados.rados.0.smithi178.stdout:316: writing smithi17829308-24 from 1015808 to 1671168 tid 2 2024-04-26T04:33:07.273 INFO:tasks.rados.rados.0.smithi178.stdout:316: writing smithi17829308-24 from 1671168 to 2359296 tid 3 2024-04-26T04:33:07.275 INFO:tasks.rados.rados.0.smithi178.stdout:316: writing smithi17829308-24 from 2359296 to 2572288 tid 4 2024-04-26T04:33:07.275 INFO:tasks.rados.rados.0.smithi178.stdout:317: rollback oid 49 current snap is 19 2024-04-26T04:33:07.275 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 49 to 18 2024-04-26T04:33:07.275 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 6 current snap is 19 2024-04-26T04:33:07.275 INFO:tasks.rados.rados.0.smithi178.stdout:318: seq_num 130 ranges {0=638976,638976=671744,1310720=802816,2113536=81920} 2024-04-26T04:33:07.280 INFO:tasks.rados.rados.0.smithi178.stdout:318: writing smithi17829308-6 from 0 to 638976 tid 1 2024-04-26T04:33:07.285 INFO:tasks.rados.rados.0.smithi178.stdout:318: writing smithi17829308-6 from 638976 to 1310720 tid 2 2024-04-26T04:33:07.291 INFO:tasks.rados.rados.0.smithi178.stdout:318: writing smithi17829308-6 from 1310720 to 2113536 tid 3 2024-04-26T04:33:07.292 INFO:tasks.rados.rados.0.smithi178.stdout:318: writing smithi17829308-6 from 2113536 to 2195456 tid 4 2024-04-26T04:33:07.292 INFO:tasks.rados.rados.0.smithi178.stdout:319: rmattr oid 13 current snap is 19 2024-04-26T04:33:07.292 INFO:tasks.rados.rados.0.smithi178.stdout:319: done (4 left) 2024-04-26T04:33:07.292 INFO:tasks.rados.rados.0.smithi178.stdout:320: read oid 18 snap -1 2024-04-26T04:33:07.293 INFO:tasks.rados.rados.0.smithi178.stdout:320: expect (ObjNum 23 snap 0 seq_num 23) 2024-04-26T04:33:07.293 INFO:tasks.rados.rados.0.smithi178.stdout:321: snap_create 2024-04-26T04:33:07.340 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:07.340 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:33:07.340 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:33:07.340 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:07.340 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.340 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:8 3.fs0 3:f50d0def:::smithi17829308-3:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:07.340 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:33:07.340 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.136075488000003, 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51428121599999999, 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 8 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:33:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200419+0000", 2024-04-26T04:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9800000000000002e-07 2024-04-26T04:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.343 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200433+0000", 2024-04-26T04:33:07.343 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4513e-05 2024-04-26T04:33:07.343 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.343 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.343 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.343 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200460+0000", 2024-04-26T04:33:07.343 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6438999999999999e-05 2024-04-26T04:33:07.343 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200477+0000", 2024-04-26T04:33:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7346e-05 2024-04-26T04:33:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200757+0000", 2024-04-26T04:33:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000279771 2024-04-26T04:33:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200766+0000", 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.7159999999999992e-06 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.341113+0000", 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14034733799999999 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.341282+0000", 2024-04-26T04:33:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000169197 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.342226+0000", 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00094361300000000005 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.343875+0000", 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016493910000000001 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714472+0000", 2024-04-26T04:33:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.37059684599999998 2024-04-26T04:33:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714699+0000", 2024-04-26T04:33:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022704800000000001 2024-04-26T04:33:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:9 3.fs0 3:f50d0def:::smithi17829308-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.135994799000002, 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71712659899999998, 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 9 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1999999999999998e-07 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200506+0000", 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.1399999999999997e-06 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200520+0000", 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4253000000000001e-05 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200531+0000", 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1375999999999999e-05 2024-04-26T04:33:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200817+0000", 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00028585699999999998 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200824+0000", 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6730000000000003e-06 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.342845+0000", 2024-04-26T04:33:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.142020961 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.343080+0000", 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000235481 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714851+0000", 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.371770347 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.715101+0000", 2024-04-26T04:33:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00025083100000000002 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917625+0000", 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20252376 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:16 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~98304,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.134837798, 2024-04-26T04:33:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.514661445, 2024-04-26T04:33:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 16 2024-04-26T04:33:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:33:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201657+0000", 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.052e-06 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201822+0000", 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016550000000000001 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201842+0000", 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0231e-05 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201856+0000", 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3234999999999999e-05 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201960+0000", 2024-04-26T04:33:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0001043 2024-04-26T04:33:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201982+0000", 2024-04-26T04:33:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1936000000000001e-05 2024-04-26T04:33:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344166+0000", 2024-04-26T04:33:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14218376399999999 2024-04-26T04:33:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344545+0000", 2024-04-26T04:33:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00037956399999999999 2024-04-26T04:33:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.346210+0000", 2024-04-26T04:33:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00166501 2024-04-26T04:33:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349256+0000", 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0030461080000000001 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716126+0000", 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.366869679 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716317+0000", 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019106599999999999 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:17 3.fs0 3:f03878c8:::smithi17829308-6:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.134609707000003, 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51482183299999995, 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 17 2024-04-26T04:33:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201885+0000", 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1060000000000001e-06 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201891+0000", 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.806e-06 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201905+0000", 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.432e-05 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201929+0000", 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4417e-05 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202056+0000", 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012684400000000001 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202066+0000", 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.5230000000000001e-06 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.347195+0000", 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.145129646 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.347408+0000", 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00021322299999999999 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.348249+0000", 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000840363 2024-04-26T04:33:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.353641+0000", 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0053921280000000004 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716520+0000", 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.36287924799999999 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716705+0000", 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018520900000000001 2024-04-26T04:33:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:18 3.fs0 3:f03878c8:::smithi17829308-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:33:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.134546579000002, 2024-04-26T04:33:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71589294699999995, 2024-04-26T04:33:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 18 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201948+0000", 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.11e-06 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201962+0000", 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4297999999999999e-05 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201990+0000", 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8243e-05 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202004+0000", 2024-04-26T04:33:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3259e-05 2024-04-26T04:33:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202132+0000", 2024-04-26T04:33:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000128596 2024-04-26T04:33:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202149+0000", 2024-04-26T04:33:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6463000000000001e-05 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349005+0000", 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14685641099999999 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349233+0000", 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000227798 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716805+0000", 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.36757169099999998 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716944+0000", 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013965200000000001 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917840+0000", 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20089542599999999 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:37 3.fs0 3:fa2b8868:::smithi17829308-13:head [append 0~212992,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.308319+0000", 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.028174225999997, 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40929669000000002, 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 37 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308319+0000", 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308319+0000", 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308321+0000", 2024-04-26T04:33:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.39e-06 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308655+0000", 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00033381099999999999 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308682+0000", 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7112999999999999e-05 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308703+0000", 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0908999999999999e-05 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349359+0000", 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.040655934999999997 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349607+0000", 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00024798700000000002 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.350900+0000", 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0012927940000000001 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.353649+0000", 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0027486519999999999 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.717506+0000", 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.36385687700000002 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.717616+0000", 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000110222 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:38 3.fs0 3:fa2b8868:::smithi17829308-13:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.308739+0000", 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.027754708000003, 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50622445199999999, 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 38 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308739+0000", 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308739+0000", 2024-04-26T04:33:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308740+0000", 2024-04-26T04:33:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7570000000000001e-06 2024-04-26T04:33:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308749+0000", 2024-04-26T04:33:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.833e-06 2024-04-26T04:33:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308772+0000", 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2779e-05 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.308787+0000", 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5532999999999998e-05 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.351662+0000", 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.042874774999999997 2024-04-26T04:33:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.351936+0000", 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00027371399999999999 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.353132+0000", 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0011959539999999999 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.354065+0000", 2024-04-26T04:33:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000933343 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.814887+0000", 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.46082132199999998 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.814963+0000", 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.6441999999999998e-05 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:346 3.fs0 3:fa2b8868:::smithi17829308-13:head [copy-from ver 14] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.181230+0000", 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 32.155263798999997, 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.80960196600000001, 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 346 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181230+0000", 2024-04-26T04:33:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181230+0000", 2024-04-26T04:33:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181232+0000", 2024-04-26T04:33:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3609999999999999e-06 2024-04-26T04:33:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181257+0000", 2024-04-26T04:33:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5100999999999999e-05 2024-04-26T04:33:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181283+0000", 2024-04-26T04:33:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5612000000000001e-05 2024-04-26T04:33:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181342+0000", 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9744000000000003e-05 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248343+0000", 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.067000834999999995 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248487+0000", 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014416900000000001 2024-04-26T04:33:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.787056+0000", 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.53856828300000004 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.789372+0000", 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0023167069999999999 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.990698+0000", 2024-04-26T04:33:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201325688 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.990831+0000", 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000133466 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:347 3.fs0 3:fa2b8868:::smithi17829308-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.181534+0000", 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 32.154959241, 2024-04-26T04:33:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.60738179000000003, 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 347 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181534+0000", 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181534+0000", 2024-04-26T04:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181535+0000", 2024-04-26T04:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0640000000000001e-06 2024-04-26T04:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181541+0000", 2024-04-26T04:33:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4099999999999999e-06 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181557+0000", 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6728999999999999e-05 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181570+0000", 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2743e-05 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248864+0000", 2024-04-26T04:33:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.067293511 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248995+0000", 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013142000000000001 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.788613+0000", 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.53961845799999997 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.788760+0000", 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014664600000000001 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.788916+0000", 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000155809 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(osd.3.22:1 3.fs0 3:f8001b32:::smithi17829308-38:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.221684+0000", 2024-04-26T04:33:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 32.114809285, 2024-04-26T04:33:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.353815977, 2024-04-26T04:33:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221684+0000", 2024-04-26T04:33:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221684+0000", 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221685+0000", 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4410000000000001e-06 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221692+0000", 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.1099999999999999e-06 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221703+0000", 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.111e-05 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221714+0000", 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1473e-05 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.448870+0000", 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.227155934 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.448943+0000", 2024-04-26T04:33:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.2631000000000003e-05 2024-04-26T04:33:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.575500+0000", 2024-04-26T04:33:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.126557278 2024-04-26T04:33:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:430 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~622592] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.182041+0000", 2024-04-26T04:33:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 21.154452272, 2024-04-26T04:33:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51141810899999995, 2024-04-26T04:33:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 430 2024-04-26T04:33:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182041+0000", 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182041+0000", 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182044+0000", 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.83e-06 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182758+0000", 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00071432400000000001 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182792+0000", 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3655999999999998e-05 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.282928+0000", 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100136005 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.283058+0000", 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013049700000000001 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.283071+0000", 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3106e-05 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.326367+0000", 2024-04-26T04:33:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.043295412999999998 2024-04-26T04:33:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.326694+0000", 2024-04-26T04:33:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00032741600000000002 2024-04-26T04:33:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.329727+0000", 2024-04-26T04:33:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0030326509999999999 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.333822+0000", 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0040948139999999996 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.693241+0000", 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.35941913199999997 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.693459+0000", 2024-04-26T04:33:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00021826500000000001 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:431 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~475136] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.282983+0000", 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 21.053509817999998, 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.411303902, 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 431 2024-04-26T04:33:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.282983+0000", 2024-04-26T04:33:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.282983+0000", 2024-04-26T04:33:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.282985+0000", 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.933e-06 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.283498+0000", 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00051254800000000004 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.283522+0000", 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4049e-05 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383665+0000", 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100142975 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383825+0000", 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015984699999999999 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384053+0000", 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022778800000000001 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.385649+0000", 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001596164 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.592570+0000", 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20692076200000001 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.694095+0000", 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10152514 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.694287+0000", 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019269600000000001 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:432 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~294912] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.383720+0000", 2024-04-26T04:33:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 20.952772991, 2024-04-26T04:33:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.352655626, 2024-04-26T04:33:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 432 2024-04-26T04:33:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383720+0000", 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383720+0000", 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383723+0000", 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7839999999999999e-06 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383953+0000", 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00023033799999999999 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.383985+0000", 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1751999999999997e-05 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384006+0000", 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.05e-05 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.386759+0000", 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0027536240000000001 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.387014+0000", 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000255002 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.388836+0000", 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001821528 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.592573+0000", 2024-04-26T04:33:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.203737154 2024-04-26T04:33:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.736257+0000", 2024-04-26T04:33:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14368439599999999 2024-04-26T04:33:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.736376+0000", 2024-04-26T04:33:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000118548 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:433 3.fs0 3:f03878c8:::smithi17829308-6:head [setxattr _header (55) in=7b] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.384045+0000", 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 20.952448681, 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.41074539900000001, 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 433 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384045+0000", 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384045+0000", 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384047+0000", 2024-04-26T04:33:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1100000000000001e-06 2024-04-26T04:33:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384056+0000", 2024-04-26T04:33:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.0100000000000001e-06 2024-04-26T04:33:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384077+0000", 2024-04-26T04:33:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1546e-05 2024-04-26T04:33:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384094+0000", 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6305999999999999e-05 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.390024+0000", 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0059306819999999996 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.390289+0000", 2024-04-26T04:33:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026486100000000002 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.391188+0000", 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00089867300000000001 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.592830+0000", 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20164169200000001 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.794632+0000", 2024-04-26T04:33:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201802486 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.794790+0000", 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000158033 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:434 3.fs0 3:f03878c8:::smithi17829308-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.384120+0000", 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 20.952373423000001, 2024-04-26T04:33:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81328199499999998, 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 434 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384120+0000", 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384120+0000", 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384121+0000", 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3069999999999999e-06 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384127+0000", 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4430000000000002e-06 2024-04-26T04:33:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384147+0000", 2024-04-26T04:33:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0533000000000001e-05 2024-04-26T04:33:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384163+0000", 2024-04-26T04:33:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5948999999999999e-05 2024-04-26T04:33:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.491986+0000", 2024-04-26T04:33:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.107822418 2024-04-26T04:33:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.492246+0000", 2024-04-26T04:33:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026025100000000003 2024-04-26T04:33:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.796841+0000", 2024-04-26T04:33:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30459499299999998 2024-04-26T04:33:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.796996+0000", 2024-04-26T04:33:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015495 2024-04-26T04:33:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:47.197402+0000", 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40040615099999999 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:471 3.fs0 3:f0b18d84:::smithi17829308-36:head [read 0~589824,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.185982+0000", 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 17.150511603999998, 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40798264899999998, 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 471 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185982+0000", 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185982+0000", 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185982+0000", 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.1300000000000001e-07 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185985+0000", 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6309999999999999e-06 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185991+0000", 2024-04-26T04:33:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8409999999999996e-06 2024-04-26T04:33:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.185995+0000", 2024-04-26T04:33:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5689999999999998e-06 2024-04-26T04:33:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.286274+0000", 2024-04-26T04:33:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10027858000000001 2024-04-26T04:33:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.286295+0000", 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1140000000000001e-05 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.390661+0000", 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10436590699999999 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.390758+0000", 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.7398999999999994e-05 2024-04-26T04:33:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.593964+0000", 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20320616899999999 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:488 3.fs0 3:f8001b32:::smithi17829308-38:head [copy-from ver 18] snapc d=[b,a] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:51.211447+0000", 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.125046769000001, 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71306741799999995, 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 488 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211447+0000", 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211447+0000", 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211449+0000", 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.554e-06 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211474+0000", 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4987999999999999e-05 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211499+0000", 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4847000000000001e-05 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211516+0000", 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6731e-05 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211811+0000", 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000294999 2024-04-26T04:33:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211828+0000", 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7842999999999998e-05 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.415705+0000", 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20387643899999999 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.416026+0000", 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00032070999999999998 2024-04-26T04:33:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.819320+0000", 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40329431799999998 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.821274+0000", 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001954258 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.924373+0000", 2024-04-26T04:33:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103099154 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.924514+0000", 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014057699999999999 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:489 3.fs0 3:f8001b32:::smithi17829308-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:51.211561+0000", 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.124932559000001, 2024-04-26T04:33:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61242426699999997, 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 489 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211561+0000", 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211561+0000", 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211561+0000", 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1699999999999997e-07 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211566+0000", 2024-04-26T04:33:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.2289999999999997e-06 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211576+0000", 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0543999999999999e-05 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211585+0000", 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1749999999999994e-06 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211947+0000", 2024-04-26T04:33:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00036154399999999999 2024-04-26T04:33:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211960+0000", 2024-04-26T04:33:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3443e-05 2024-04-26T04:33:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.615185+0000", 2024-04-26T04:33:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40322487499999998 2024-04-26T04:33:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.615237+0000", 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.2206999999999997e-05 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.823589+0000", 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.208351707 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.823795+0000", 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00020569300000000001 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.823985+0000", 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019013299999999999 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(osd.11.22:7 3.fs0 3:fcc81d86:::smithi17829308-46:head [assert-version v18,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:51.417382+0000", 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.919111744, 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40038985500000002, 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.417382+0000", 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.417382+0000", 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.417384+0000", 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3829999999999999e-06 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.417394+0000", 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8090000000000004e-06 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.417414+0000", 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.065e-05 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.417433+0000", 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.823e-05 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.615296+0000", 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.197863866 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.615368+0000", 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1185000000000004e-05 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.817771+0000", 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.202403732 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:501 3.fs0 3:f8001b32:::smithi17829308-38:head [copy-from ver 2] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:52.384346+0000", 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 14.952146952, 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.611539218, 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 501 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384346+0000", 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384346+0000", 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384349+0000", 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3760000000000002e-06 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384360+0000", 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1240000000000001e-05 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384436+0000", 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5871999999999995e-05 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384464+0000", 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7693000000000001e-05 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.395288+0000", 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.010824926 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.395451+0000", 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000162607 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.699338+0000", 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.303887405 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.792390+0000", 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.093051153999999997 2024-04-26T04:33:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.995723+0000", 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.203333075 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.995886+0000", 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016286999999999999 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.454 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:07.455 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T04:33:07.456 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:07.454+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:07.521 INFO:tasks.rados.rados.0.smithi178.stdout:314: finishing write tid 3 to smithi17829308-42 2024-04-26T04:33:07.555 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:07.552+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:07.622 INFO:tasks.rados.rados.0.smithi178.stdout:314: finishing write tid 4 to smithi17829308-42 2024-04-26T04:33:07.635 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 72/30 ECSubWriteReply(tid=108, last_complete=72'77, committed=1, applied=1))", 2024-04-26T04:33:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.581068+0000", 2024-04-26T04:33:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.053076301999999999, 2024-04-26T04:33:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.053114007999999997, 2024-04-26T04:33:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.581068+0000", 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.581068+0000", 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.581069+0000", 2024-04-26T04:33:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.7700000000000005e-07 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.581072+0000", 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8909999999999999e-06 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.581081+0000", 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1670000000000005e-06 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.581087+0000", 2024-04-26T04:33:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.4359999999999999e-06 2024-04-26T04:33:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:809 3.5s0 3:ac36f45f:::smithi17829308-24:head [append 0~655360 in=655360b] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.364556+0000", 2024-04-26T04:33:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.26958845599999998, 2024-04-26T04:33:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.26972625700000002, 2024-04-26T04:33:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 809 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.364556+0000", 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.364556+0000", 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.364559+0000", 2024-04-26T04:33:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6639999999999998e-06 2024-04-26T04:33:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365107+0000", 2024-04-26T04:33:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00054841600000000003 2024-04-26T04:33:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365118+0000", 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0607e-05 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365124+0000", 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.5729999999999996e-06 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365213+0000", 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8837000000000003e-05 2024-04-26T04:33:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365218+0000", 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.7700000000000001e-06 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.470522+0000", 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10530410699999999 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.470832+0000", 2024-04-26T04:33:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00030972200000000001 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.472888+0000", 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0020565230000000002 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.477500+0000", 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0046116150000000003 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:810 3.5s0 3:ac36f45f:::smithi17829308-24:head [append 0~688128 in=688128b] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.365138+0000", 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.26900687499999998, 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.26929492999999999, 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 810 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365138+0000", 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365138+0000", 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365138+0000", 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.2900000000000004e-07 2024-04-26T04:33:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365458+0000", 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00032031300000000002 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365464+0000", 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.7440000000000004e-06 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365468+0000", 2024-04-26T04:33:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.6320000000000001e-06 2024-04-26T04:33:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365553+0000", 2024-04-26T04:33:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4975999999999994e-05 2024-04-26T04:33:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365558+0000", 2024-04-26T04:33:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0499999999999999e-06 2024-04-26T04:33:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.474113+0000", 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.108555369 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.474311+0000", 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019815800000000001 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.476043+0000", 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0017315309999999999 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.579516+0000", 2024-04-26T04:33:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103473022 2024-04-26T04:33:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:811 3.5s0 3:ac36f45f:::smithi17829308-24:head [append 0~212992 in=212992b] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.365476+0000", 2024-04-26T04:33:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.26866865499999998, 2024-04-26T04:33:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.269099645, 2024-04-26T04:33:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 811 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365476+0000", 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365476+0000", 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365476+0000", 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7800000000000002e-07 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365576+0000", 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9685000000000005e-05 2024-04-26T04:33:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365581+0000", 2024-04-26T04:33:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.7849999999999999e-06 2024-04-26T04:33:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365584+0000", 2024-04-26T04:33:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.4750000000000002e-06 2024-04-26T04:33:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365631+0000", 2024-04-26T04:33:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.6653000000000002e-05 2024-04-26T04:33:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365635+0000", 2024-04-26T04:33:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3730000000000003e-06 2024-04-26T04:33:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.577379+0000", 2024-04-26T04:33:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21174357399999999 2024-04-26T04:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.577489+0000", 2024-04-26T04:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000110707 2024-04-26T04:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.578124+0000", 2024-04-26T04:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00063492300000000002 2024-04-26T04:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.579553+0000", 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0014284499999999999 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:812 3.5s0 3:ac36f45f:::smithi17829308-24:head [setxattr _header (57) in=64b] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.365591+0000", 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.26855315299999999, 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.26912366399999998, 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 812 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365591+0000", 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365591+0000", 2024-04-26T04:33:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365592+0000", 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.6600000000000002e-07 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365593+0000", 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8080000000000001e-06 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365598+0000", 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3259999999999997e-06 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365601+0000", 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.4589999999999998e-06 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365658+0000", 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.6523000000000001e-05 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365661+0000", 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.704e-06 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.578678+0000", 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21301682399999999 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.578764+0000", 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.5344000000000001e-05 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.579203+0000", 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000439772 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580141+0000", 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00093775900000000003 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:813 3.5s0 3:ac36f45f:::smithi17829308-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.365605+0000", 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.268539044, 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.26924663999999998, 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "waiting for rw locks", 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 813 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365605+0000", 2024-04-26T04:33:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365605+0000", 2024-04-26T04:33:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365606+0000", 2024-04-26T04:33:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9499999999999998e-07 2024-04-26T04:33:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365607+0000", 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.339e-06 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365612+0000", 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.532e-06 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365615+0000", 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.191e-06 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.366700+0000", 2024-04-26T04:33:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0010852710000000001 2024-04-26T04:33:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.366706+0000", 2024-04-26T04:33:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.2230000000000004e-06 2024-04-26T04:33:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580054+0000", 2024-04-26T04:33:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21334779800000001 2024-04-26T04:33:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580129+0000", 2024-04-26T04:33:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5333000000000005e-05 2024-04-26T04:33:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 72/30 ECSubWriteReply(tid=105, last_complete=72'74, committed=1, applied=1))", 2024-04-26T04:33:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.576324+0000", 2024-04-26T04:33:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.057820024999999997, 2024-04-26T04:33:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.058648532000000003, 2024-04-26T04:33:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "no flag points reached", 2024-04-26T04:33:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.576324+0000", 2024-04-26T04:33:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.576324+0000", 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.576325+0000", 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4600000000000001e-07 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.576326+0000", 2024-04-26T04:33:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.421e-06 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.576330+0000", 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.1359999999999999e-06 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 72/30 ECSubWriteReply(tid=108, last_complete=72'77, committed=1, applied=1))", 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.580772+0000", 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.053372438000000001, 2024-04-26T04:33:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.054264338000000002, 2024-04-26T04:33:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580772+0000", 2024-04-26T04:33:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580772+0000", 2024-04-26T04:33:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580773+0000", 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.2699999999999999e-07 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580777+0000", 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.4000000000000002e-06 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580791+0000", 2024-04-26T04:33:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3910999999999999e-05 2024-04-26T04:33:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580799+0000", 2024-04-26T04:33:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.4919999999999996e-06 2024-04-26T04:33:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 72/30 ECSubWriteReply(tid=107, last_complete=72'76, committed=1, applied=1))", 2024-04-26T04:33:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.581022+0000", 2024-04-26T04:33:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.053122535999999998, 2024-04-26T04:33:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.054085164999999998, 2024-04-26T04:33:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.581022+0000", 2024-04-26T04:33:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.581022+0000", 2024-04-26T04:33:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.581023+0000", 2024-04-26T04:33:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5199999999999996e-07 2024-04-26T04:33:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.581027+0000", 2024-04-26T04:33:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.1899999999999997e-06 2024-04-26T04:33:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.581040+0000", 2024-04-26T04:33:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3553999999999999e-05 2024-04-26T04:33:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.581048+0000", 2024-04-26T04:33:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.9160000000000001e-06 2024-04-26T04:33:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:07.675 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.675 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:07.675 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-04-26T04:33:07.675 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 9 2024-04-26T04:33:07.675 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:07.676 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T04:33:07.677 INFO:tasks.rados.rados.0.smithi178.stdout:316: finishing write tid 1 to smithi17829308-24 2024-04-26T04:33:07.677 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:07.673+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:07.718 INFO:tasks.rados.rados.0.smithi178.stdout:320: expect (ObjNum 23 snap 0 seq_num 23) 2024-04-26T04:33:07.747 INFO:tasks.rados.rados.0.smithi178.stdout:314: finishing write tid 5 to smithi17829308-42 2024-04-26T04:33:07.778 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:07.776+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:07.781 INFO:tasks.rados.rados.0.smithi178.stdout:316: finishing write tid 2 to smithi17829308-24 2024-04-26T04:33:07.791 INFO:tasks.rados.rados.0.smithi178.stdout:318: finishing write tid 1 to smithi17829308-6 2024-04-26T04:33:07.791 INFO:tasks.rados.rados.0.smithi178.stdout:318: finishing write tid 2 to smithi17829308-6 2024-04-26T04:33:07.842 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:33:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:33:07.843 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:07.857 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T04:33:07.878 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:07.876+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:07.881 INFO:tasks.rados.rados.0.smithi178.stdout:316: finishing write tid 3 to smithi17829308-24 2024-04-26T04:33:07.882 INFO:tasks.rados.rados.0.smithi178.stdout:316: finishing write tid 4 to smithi17829308-24 2024-04-26T04:33:07.883 INFO:tasks.rados.rados.0.smithi178.stdout:316: finishing write tid 5 to smithi17829308-24 2024-04-26T04:33:07.892 INFO:tasks.rados.rados.0.smithi178.stdout:318: finishing write tid 3 to smithi17829308-6 2024-04-26T04:33:07.927 INFO:tasks.rados.rados.0.smithi178.stdout:316: finishing write tid 6 to smithi17829308-24 2024-04-26T04:33:07.927 INFO:tasks.rados.rados.0.smithi178.stdout:316: oid 24 updating version 0 to 73 2024-04-26T04:33:07.927 INFO:tasks.rados.rados.0.smithi178.stdout:316: oid 24 updating version 73 to 76 2024-04-26T04:33:07.927 INFO:tasks.rados.rados.0.smithi178.stdout:316: oid 24 version 76 is already newer than 75 2024-04-26T04:33:07.927 INFO:tasks.rados.rados.0.smithi178.stdout:316: oid 24 updating version 76 to 77 2024-04-26T04:33:07.927 INFO:tasks.rados.rados.0.smithi178.stdout:316: oid 24 version 77 is already newer than 74 2024-04-26T04:33:07.927 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 24 v 77 (ObjNum 129 snap 19 seq_num 129) dirty exists 2024-04-26T04:33:07.927 INFO:tasks.rados.rados.0.smithi178.stdout:316: left oid 24 (ObjNum 129 snap 19 seq_num 129) 2024-04-26T04:33:07.980 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:07.977+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:07.992 INFO:tasks.rados.rados.0.smithi178.stdout:318: finishing write tid 4 to smithi17829308-6 2024-04-26T04:33:07.993 INFO:tasks.rados.rados.0.smithi178.stdout:318: finishing write tid 5 to smithi17829308-6 2024-04-26T04:33:07.999 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:33:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:33:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:21 3.5s0 3:a4a2788c:::smithi17829308-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:33:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.692893929, 2024-04-26T04:33:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71502945500000004, 2024-04-26T04:33:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 21 2024-04-26T04:33:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:33:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:33:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304624+0000", 2024-04-26T04:33:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.186e-06 2024-04-26T04:33:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304629+0000", 2024-04-26T04:33:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3630000000000004e-06 2024-04-26T04:33:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304643+0000", 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4800000000000001e-05 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304654+0000", 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0665999999999999e-05 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507573+0000", 2024-04-26T04:33:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20291931099999999 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507644+0000", 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1131999999999996e-05 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.816490+0000", 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30884542700000001 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.816596+0000", 2024-04-26T04:33:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000105678 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.019652+0000", 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20305689199999999 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:25 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~147456,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.692769200999997, 2024-04-26T04:33:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61210120899999998, 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 25 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304749+0000", 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.277e-06 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304925+0000", 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017555399999999999 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304943+0000", 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8529000000000002e-05 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304954+0000", 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0879999999999999e-05 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608582+0000", 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30362831899999998 2024-04-26T04:33:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608711+0000", 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012887 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609284+0000", 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00057331000000000005 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.710054+0000", 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100769761 2024-04-26T04:33:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.916732+0000", 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20667817799999999 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.916849+0000", 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000116531 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:26 3.5s0 3:afb9923e:::smithi17829308-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.692547398999999, 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71271566600000003, 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 26 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.2399999999999996e-07 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304975+0000", 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.6460000000000001e-06 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304984+0000", 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1540000000000008e-06 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304992+0000", 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5259999999999996e-06 2024-04-26T04:33:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609850+0000", 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.304857769 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609963+0000", 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011346899999999999 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.610506+0000", 2024-04-26T04:33:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00054322400000000003 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.710058+0000", 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.099551985999999995 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017545+0000", 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30748642599999998 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017685+0000", 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014054200000000001 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:360 3.5s0 3:a44e1dec:::smithi17829308-26:head [rollback 2] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.340709+0000", 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 32.656807975, 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.54345505299999997, 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 360 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340709+0000", 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340709+0000", 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340710+0000", 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9100000000000003e-07 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340714+0000", 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.1760000000000003e-06 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340725+0000", 2024-04-26T04:33:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1664e-05 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340734+0000", 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8899999999999996e-06 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340844+0000", 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000110005 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340851+0000", 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7830000000000001e-06 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.577334+0000", 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23648275699999999 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.577565+0000", 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00023120000000000001 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579037+0000", 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00147218 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.682611+0000", 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103574176 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.884048+0000", 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201436371 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.884164+0000", 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011616 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:361 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.340770+0000", 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 32.656746755, 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.644132284, 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 361 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340770+0000", 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340770+0000", 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340771+0000", 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8400000000000004e-07 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340774+0000", 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.349e-06 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340783+0000", 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.3470000000000005e-06 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340792+0000", 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8740000000000001e-06 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340874+0000", 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.1458000000000005e-05 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340879+0000", 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4110000000000004e-06 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579882+0000", 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23900315899999999 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.580145+0000", 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026314300000000002 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.581275+0000", 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001129997 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.682889+0000", 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101613946 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.984790+0000", 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30190081600000002 2024-04-26T04:33:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.984902+0000", 2024-04-26T04:33:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0001122 2024-04-26T04:33:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:478 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~688128] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.219546+0000", 2024-04-26T04:33:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 17.777970615000001, 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.60491518499999997, 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 478 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219546+0000", 2024-04-26T04:33:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219546+0000", 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219549+0000", 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.322e-06 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219983+0000", 2024-04-26T04:33:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00043452399999999999 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219998+0000", 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5231999999999999e-05 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.220007+0000", 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9379999999999998e-06 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.320021+0000", 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100013729 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.320111+0000", 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9907999999999994e-05 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.320914+0000", 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00080305000000000003 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427530+0000", 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.106616091 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.824390+0000", 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.396859621 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.824461+0000", 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1769999999999999e-05 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:479 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~81920] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.221012+0000", 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 17.776505263000001, 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.70366377099999999, 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 479 2024-04-26T04:33:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221012+0000", 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221012+0000", 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221013+0000", 2024-04-26T04:33:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4750000000000001e-06 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221057+0000", 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.4305999999999999e-05 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221069+0000", 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1783000000000001e-05 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221077+0000", 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7649999999999993e-06 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321785+0000", 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100707597 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.322104+0000", 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00031923599999999998 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.323441+0000", 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001337233 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427532+0000", 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10409099199999999 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924579+0000", 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49704692 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924675+0000", 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6464000000000003e-05 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:480 3.5s0 3:afb9923e:::smithi17829308-9:head [setxattr _header (56) in=7b] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 17.776427374000001, 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.70383227299999995, 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.039 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.039 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.039 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 480 2024-04-26T04:33:08.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.039 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:33:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:33:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9999999999999997e-07 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221093+0000", 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.241e-06 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221100+0000", 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6080000000000001e-06 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221106+0000", 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4999999999999999e-06 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.324423+0000", 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103317706 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.324652+0000", 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022915899999999999 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.325569+0000", 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000916941 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427565+0000", 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101995135 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924820+0000", 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49725566500000001 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924922+0000", 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000101618 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:481 3.5s0 3:afb9923e:::smithi17829308-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.221113+0000", 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 17.776403600999998, 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.80581624500000004, 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 481 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221113+0000", 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221113+0000", 2024-04-26T04:33:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221114+0000", 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.5700000000000002e-07 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221117+0000", 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.552e-06 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221125+0000", 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7710000000000006e-06 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321214+0000", 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100088539 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.326595+0000", 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0053815479999999999 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.326744+0000", 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014871700000000001 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.925438+0000", 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.59869437999999997 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.925572+0000", 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000133912 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.026930+0000", 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101357269 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:728 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~475136] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.306711+0000", 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.6908063929999999, 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.90587526100000004, 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 728 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.306711+0000", 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.306711+0000", 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.306713+0000", 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8389999999999998e-06 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.307016+0000", 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00030270699999999999 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.307050+0000", 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.4099e-05 2024-04-26T04:33:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.307069+0000", 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.836e-05 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396493+0000", 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.089424218999999999 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396500+0000", 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.6960000000000005e-06 2024-04-26T04:33:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.597275+0000", 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20077457100000001 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.597440+0000", 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000165021 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.599124+0000", 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001683882 2024-04-26T04:33:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703777+0000", 2024-04-26T04:33:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.104652914 2024-04-26T04:33:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.212458+0000", 2024-04-26T04:33:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50868092200000004 2024-04-26T04:33:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.212586+0000", 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012803099999999999 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:729 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~655360] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.316292+0000", 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.6812244310000004, 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.89658193900000005, 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 729 2024-04-26T04:33:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316292+0000", 2024-04-26T04:33:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316292+0000", 2024-04-26T04:33:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316295+0000", 2024-04-26T04:33:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4490000000000002e-06 2024-04-26T04:33:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316680+0000", 2024-04-26T04:33:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00038542699999999998 2024-04-26T04:33:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316716+0000", 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.5405999999999999e-05 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316736+0000", 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0505000000000001e-05 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396572+0000", 2024-04-26T04:33:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.079835357999999995 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396575+0000", 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.529e-06 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.600521+0000", 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20394542500000001 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.600677+0000", 2024-04-26T04:33:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015607699999999999 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.602205+0000", 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001528661 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703787+0000", 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10158194500000001 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.212771+0000", 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50898380499999996 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.212874+0000", 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000103352 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:730 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~622592] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.325217+0000", 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.6723003890000001, 2024-04-26T04:33:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.88791738899999995, 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 730 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325217+0000", 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325217+0000", 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325219+0000", 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.283e-06 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325555+0000", 2024-04-26T04:33:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00033651699999999999 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325585+0000", 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9173999999999999e-05 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325601+0000", 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6551000000000001e-05 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396592+0000", 2024-04-26T04:33:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.070990756000000002 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396595+0000", 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.061e-06 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.603348+0000", 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.206753411 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.603532+0000", 2024-04-26T04:33:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018416400000000001 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.605092+0000", 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001559337 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703790+0000", 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.098698498999999995 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213032+0000", 2024-04-26T04:33:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50924209399999998 2024-04-26T04:33:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213134+0000", 2024-04-26T04:33:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000101542 2024-04-26T04:33:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:731 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~278528] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.329197+0000", 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.6683194409999995, 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.88429409199999998, 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 731 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329197+0000", 2024-04-26T04:33:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329197+0000", 2024-04-26T04:33:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329199+0000", 2024-04-26T04:33:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5910000000000001e-06 2024-04-26T04:33:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329372+0000", 2024-04-26T04:33:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017300599999999999 2024-04-26T04:33:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329398+0000", 2024-04-26T04:33:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6400000000000001e-05 2024-04-26T04:33:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329413+0000", 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4548e-05 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396610+0000", 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.067197398000000005 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396613+0000", 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.033e-06 2024-04-26T04:33:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.606931+0000", 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21031707599999999 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.607074+0000", 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000143144 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.608197+0000", 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0011229980000000001 2024-04-26T04:33:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703793+0000", 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.095596134999999999 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213368+0000", 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50957523800000004 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213492+0000", 2024-04-26T04:33:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012352500000000001 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:732 3.5s0 3:a44e1dec:::smithi17829308-26:head [setxattr _header (57) in=7b] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.329456+0000", 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.6680607849999998, 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.8844417, 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 732 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329456+0000", 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329456+0000", 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329457+0000", 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1969999999999999e-06 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329465+0000", 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1969999999999999e-06 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329483+0000", 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8351000000000001e-05 2024-04-26T04:33:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329497+0000", 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4114000000000001e-05 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.407278+0000", 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.077780571000000007 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.407289+0000", 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.111e-05 2024-04-26T04:33:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.609056+0000", 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201767264 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.609198+0000", 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014221700000000001 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.609956+0000", 2024-04-26T04:33:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00075800100000000001 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703800+0000", 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.093843736999999997 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213750+0000", 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50995032600000001 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213898+0000", 2024-04-26T04:33:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000147615 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:733 3.5s0 3:a44e1dec:::smithi17829308-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.329515+0000", 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.6680018890000001, 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0903875629999999, 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 733 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329515+0000", 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329515+0000", 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329516+0000", 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.2299999999999999e-07 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329520+0000", 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5399999999999997e-06 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329537+0000", 2024-04-26T04:33:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6393999999999999e-05 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329550+0000", 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2934e-05 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.407334+0000", 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.077784331999999998 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.407342+0000", 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.9699999999999999e-06 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703462+0000", 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29612015899999999 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703734+0000", 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00027200199999999998 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.214603+0000", 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51086900899999999 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.214847+0000", 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00024385300000000001 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.419903+0000", 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.205055447 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(osd.5.23:4 3.5s0 3:afb9923e:::smithi17829308-9:head [assert-version v44,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-26T04:33:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:03.630782+0000", 2024-04-26T04:33:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.366735394, 2024-04-26T04:33:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.60668015399999997, 2024-04-26T04:33:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630782+0000", 2024-04-26T04:33:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630782+0000", 2024-04-26T04:33:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630785+0000", 2024-04-26T04:33:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3270000000000001e-06 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630798+0000", 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2893e-05 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630832+0000", 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3865999999999997e-05 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630853+0000", 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1437000000000002e-05 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.833963+0000", 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20310972999999999 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.834253+0000", 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00029035900000000002 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:04.237462+0000", 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.403208542 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:810 3.5s0 3:ac36f45f:::smithi17829308-24:head [append 0~688128 in=688128b] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.365138+0000", 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.63237934799999995, 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51736411000000004, 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 810 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365138+0000", 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365138+0000", 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365138+0000", 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.2900000000000004e-07 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365458+0000", 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00032031300000000002 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365464+0000", 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.7440000000000004e-06 2024-04-26T04:33:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365468+0000", 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.6320000000000001e-06 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365553+0000", 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4975999999999994e-05 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365558+0000", 2024-04-26T04:33:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0499999999999999e-06 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.474113+0000", 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.108555369 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.474311+0000", 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019815800000000001 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.476043+0000", 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0017315309999999999 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.579516+0000", 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103473022 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.882375+0000", 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30285874099999999 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.882502+0000", 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000127045 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:811 3.5s0 3:ac36f45f:::smithi17829308-24:head [append 0~212992 in=212992b] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.365476+0000", 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.63204112800000001, 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51761272800000002, 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 811 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365476+0000", 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365476+0000", 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365476+0000", 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7800000000000002e-07 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365576+0000", 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9685000000000005e-05 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365581+0000", 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.7849999999999999e-06 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365584+0000", 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.4750000000000002e-06 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365631+0000", 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.6653000000000002e-05 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365635+0000", 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3730000000000003e-06 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.577379+0000", 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21174357399999999 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.577489+0000", 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000110707 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.578124+0000", 2024-04-26T04:33:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00063492300000000002 2024-04-26T04:33:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.579553+0000", 2024-04-26T04:33:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0014284499999999999 2024-04-26T04:33:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.882980+0000", 2024-04-26T04:33:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30342730600000001 2024-04-26T04:33:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.883089+0000", 2024-04-26T04:33:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010841900000000001 2024-04-26T04:33:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:812 3.5s0 3:ac36f45f:::smithi17829308-24:head [setxattr _header (57) in=64b] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.365591+0000", 2024-04-26T04:33:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.63192562600000002, 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51816032899999998, 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 812 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365591+0000", 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365591+0000", 2024-04-26T04:33:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365592+0000", 2024-04-26T04:33:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.6600000000000002e-07 2024-04-26T04:33:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365593+0000", 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8080000000000001e-06 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365598+0000", 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3259999999999997e-06 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365601+0000", 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.4589999999999998e-06 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365658+0000", 2024-04-26T04:33:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.6523000000000001e-05 2024-04-26T04:33:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365661+0000", 2024-04-26T04:33:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.704e-06 2024-04-26T04:33:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.578678+0000", 2024-04-26T04:33:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21301682399999999 2024-04-26T04:33:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.578764+0000", 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.5344000000000001e-05 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.579203+0000", 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000439772 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580141+0000", 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00093775900000000003 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.883589+0000", 2024-04-26T04:33:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30344762400000003 2024-04-26T04:33:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.883752+0000", 2024-04-26T04:33:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016281999999999999 2024-04-26T04:33:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:813 3.5s0 3:ac36f45f:::smithi17829308-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.365605+0000", 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.63191151700000003, 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.56179527200000001, 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 813 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365605+0000", 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365605+0000", 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365606+0000", 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9499999999999998e-07 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365607+0000", 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.339e-06 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365612+0000", 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.532e-06 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365615+0000", 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.191e-06 2024-04-26T04:33:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.366700+0000", 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0010852710000000001 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.366706+0000", 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.2230000000000004e-06 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580054+0000", 2024-04-26T04:33:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21334779800000001 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580129+0000", 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5333000000000005e-05 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.884613+0000", 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30448329899999999 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.884873+0000", 2024-04-26T04:33:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026001900000000001 2024-04-26T04:33:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.927401+0000", 2024-04-26T04:33:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.042527971999999997 2024-04-26T04:33:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:08.114 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:08.116 INFO:tasks.rados.rados.0.smithi178.stdout:317: finishing rollback tid 0 to smithi17829308-49 2024-04-26T04:33:08.116 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T04:33:08.117 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:08.113+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:08.128 INFO:tasks.rados.rados.0.smithi178.stdout:317: finishing rollback tid 1 to smithi17829308-49 2024-04-26T04:33:08.197 INFO:tasks.rados.rados.0.smithi178.stdout:318: finishing write tid 6 to smithi17829308-6 2024-04-26T04:33:08.197 INFO:tasks.rados.rados.0.smithi178.stdout:318: oid 6 updating version 0 to 84 2024-04-26T04:33:08.197 INFO:tasks.rados.rados.0.smithi178.stdout:318: oid 6 version 84 is already newer than 82 2024-04-26T04:33:08.197 INFO:tasks.rados.rados.0.smithi178.stdout:318: oid 6 updating version 84 to 85 2024-04-26T04:33:08.197 INFO:tasks.rados.rados.0.smithi178.stdout:318: oid 6 version 85 is already newer than 83 2024-04-26T04:33:08.197 INFO:tasks.rados.rados.0.smithi178.stdout:318: oid 6 updating version 85 to 86 2024-04-26T04:33:08.197 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 6 v 86 (ObjNum 130 snap 19 seq_num 130) dirty exists 2024-04-26T04:33:08.197 INFO:tasks.rados.rados.0.smithi178.stdout:318: left oid 6 (ObjNum 130 snap 19 seq_num 130) 2024-04-26T04:33:08.218 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:08.217+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:08.230 INFO:tasks.rados.rados.0.smithi178.stdout:314: finishing write tid 6 to smithi17829308-42 2024-04-26T04:33:08.230 INFO:tasks.rados.rados.0.smithi178.stdout:314: oid 42 updating version 0 to 57 2024-04-26T04:33:08.231 INFO:tasks.rados.rados.0.smithi178.stdout:314: oid 42 version 57 is already newer than 54 2024-04-26T04:33:08.231 INFO:tasks.rados.rados.0.smithi178.stdout:314: oid 42 version 57 is already newer than 56 2024-04-26T04:33:08.231 INFO:tasks.rados.rados.0.smithi178.stdout:314: oid 42 version 57 is already newer than 53 2024-04-26T04:33:08.231 INFO:tasks.rados.rados.0.smithi178.stdout:314: oid 42 version 57 is already newer than 55 2024-04-26T04:33:08.231 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 42 v 57 (ObjNum 128 snap 18 seq_num 128) dirty exists 2024-04-26T04:33:08.231 INFO:tasks.rados.rados.0.smithi178.stdout:314: left oid 42 (ObjNum 128 snap 18 seq_num 128) 2024-04-26T04:33:08.247 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:08.247 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:33:08.248 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-04-26T04:33:08.248 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:08.254 INFO:teuthology.orchestra.run.smithi012.stderr:set 3.a pg_upmap mapping to [3,1,7,13,6,9,8] 2024-04-26T04:33:08.254 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-04-26T04:33:08.260 INFO:tasks.rados.rados.0.smithi178.stdout:314: done (5 left) 2024-04-26T04:33:08.260 INFO:tasks.rados.rados.0.smithi178.stdout:316: done (4 left) 2024-04-26T04:33:08.261 INFO:tasks.rados.rados.0.smithi178.stdout:318: done (3 left) 2024-04-26T04:33:08.261 INFO:tasks.rados.rados.0.smithi178.stdout:320: done (2 left) 2024-04-26T04:33:08.261 INFO:tasks.rados.rados.0.smithi178.stdout:321: done (1 left) 2024-04-26T04:33:08.261 INFO:tasks.rados.rados.0.smithi178.stdout:322: delete oid 47 current snap is 20 2024-04-26T04:33:08.265 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T04:33:08.318 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:08.320+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:08.334 INFO:tasks.rados.rados.0.smithi178.stdout:317: finishing rollback tid 2 to smithi17829308-49 2024-04-26T04:33:08.334 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 49 v 61 (ObjNum 117 snap 16 seq_num 117) dirty exists 2024-04-26T04:33:08.419 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:08.417+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:33:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-04-26T04:33:08.434 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-04-26T04:33:08.435 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:08.448 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T04:33:08.520 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:08.517+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:08.574 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-04-26T04:33:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-04-26T04:33:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:33:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:320 3.6s0 3:647b920d:::smithi17829308-33:head [create,setxattr eROReghfPdlf^m`e`^hhNZXOkdsQ]]kX`senUiSZ (875) in=40b,setxattr s`rNbaqcP\\diTaoZYXg[m[inZ`]QtZdjom`_dhUX (1928) in=40b,setxattr [[YXf (69) in=5b,setxattr ]rjrtd]YkcY\\lQ[bOp_lNWs\\NOirOQ[Ri`rndifj (5260) in=40b,setxattr n[ZSX[Rc`\\dlYpV^rPdsQfnbg^l]ioSo^`oZsXaf (19016) in=40b,setxattr b\\ffPRnhZeR\\shWQYPRaZVroNXYoUYf[eemPeQen (1648) in=40b,setxattr [gNas\\_VhjX_YRX[aQbekgbWYg`P_SWgskeiZbkP (1333) in=40b,setxattr Ysi^k[[WWsYagaeecamn^bcecbfS]p^i^jfgsmTY (6736) in=40b,setxattr Nm_N]c]XYfmpWd\\aiW^bPSdY]TjNrnirngZdd_jb (103) in=40b,setxattr cisVjO^iia\\[tQaXjaroW`VRNl[aiYR\\tcnhVsfb (8841) in=40b,setxattr Zgg]\\aOQfmaVkY (69) in=14b,setxattr X`i_`tsrZ[XRfqacNap`[O`nSf]O`[Z[Oc[aenTc (1647) in=40b,setxattr oV[OofncW[WdO[XrprmdpkoptWZRfcPm^Rkernnf (34) in=40b,setxattr \\UQYgqaahhcRrkP`h`SgamahYafPmsojP_snXkmc (314) in=40b,setxattr pjilgXb]Qkl]gt__a_ZrtRptOPONOVrsmqcZ__^T (840) in=40b,setxattr Wa`ZaT[b]kbqdZrfSUkVafjqXod\\qd (69) in=30b,setxattr [YaXgtehetrqtspbYYY[ZqritVsbZeW[ak`ma``l (418) in=40b,setxattr hPanNtTRrfi`tYcdO]eRl]`fj`hllRNnXThl\\YO (69) in=39b,setxattr S_`Ri[Qj]fZqXkqqZYcte[N]Wq\\cQ]PlQdllcb]s (1542) in=40b,setxattr ttkZc`XVXmWiqdYWeagrmr^i[OUs^jpafRbOUWdp (594) in=40b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:34.392445+0000", 2024-04-26T04:33:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 34.176632521999998, 2024-04-26T04:33:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.331045494, 2024-04-26T04:33:08.574 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 320 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392445+0000", 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392445+0000", 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392450+0000", 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6569999999999996e-06 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392623+0000", 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017330500000000001 2024-04-26T04:33:08.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392664+0000", 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0399999999999999e-05 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.492771+0000", 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10010693 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.520054+0000", 2024-04-26T04:33:08.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.027283827 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.520231+0000", 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017687399999999999 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.521930+0000", 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0016988629999999999 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.722167+0000", 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20023658899999999 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.722468+0000", 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030164599999999998 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.723491+0000", 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010224030000000001 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:340 3.6s0 3:674cc614:::smithi17829308-50:head [append 0~573440 in=573440b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.177236+0000", 2024-04-26T04:33:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 33.391841366000001, 2024-04-26T04:33:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50180553100000003, 2024-04-26T04:33:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.580 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 340 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.177236+0000", 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.177236+0000", 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.177239+0000", 2024-04-26T04:33:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5100000000000001e-06 2024-04-26T04:33:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178007+0000", 2024-04-26T04:33:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00076857200000000003 2024-04-26T04:33:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.582 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178041+0000", 2024-04-26T04:33:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3909999999999999e-05 2024-04-26T04:33:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178057+0000", 2024-04-26T04:33:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5906999999999999e-05 2024-04-26T04:33:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178187+0000", 2024-04-26T04:33:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012976099999999999 2024-04-26T04:33:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178351+0000", 2024-04-26T04:33:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016396800000000001 2024-04-26T04:33:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.179822+0000", 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001470507 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.185617+0000", 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0057954089999999996 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.678895+0000", 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49327816899999999 2024-04-26T04:33:08.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679042+0000", 2024-04-26T04:33:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000146818 2024-04-26T04:33:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:341 3.6s0 3:674cc614:::smithi17829308-50:head [append 0~196608 in=196608b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180077+0000", 2024-04-26T04:33:08.586 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 33.389000301999999, 2024-04-26T04:33:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49926801199999998, 2024-04-26T04:33:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 341 2024-04-26T04:33:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.587 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180077+0000", 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180077+0000", 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180081+0000", 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2459999999999999e-06 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180497+0000", 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00041676000000000001 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180529+0000", 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1497e-05 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180556+0000", 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6594e-05 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.181275+0000", 2024-04-26T04:33:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00071977499999999997 2024-04-26T04:33:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.181487+0000", 2024-04-26T04:33:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211855 2024-04-26T04:33:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.182884+0000", 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00139722 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287222+0000", 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10433723 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679247+0000", 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.392025711 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679345+0000", 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8123999999999998e-05 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:342 3.6s0 3:674cc614:::smithi17829308-50:head [setxattr _header (56) in=63b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180608+0000", 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 33.388469667999999, 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49908019100000001, 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.592 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 342 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180608+0000", 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180608+0000", 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180610+0000", 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8029999999999999e-06 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180619+0000", 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3170000000000008e-06 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180642+0000", 2024-04-26T04:33:08.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2442999999999998e-05 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180658+0000", 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6779999999999999e-05 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.183978+0000", 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003319598 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.184127+0000", 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014915100000000001 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.185029+0000", 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00090211999999999996 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287425+0000", 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10239564600000001 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679540+0000", 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39211458199999999 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679688+0000", 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000148751 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:343 3.6s0 3:674cc614:::smithi17829308-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180684+0000", 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 33.388393780999998, 2024-04-26T04:33:08.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.71138035399999999, 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 343 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180684+0000", 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180684+0000", 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180685+0000", 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2869999999999999e-06 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180691+0000", 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0929999999999999e-06 2024-04-26T04:33:08.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180712+0000", 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0514e-05 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180728+0000", 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5665999999999999e-05 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.286928+0000", 2024-04-26T04:33:08.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.106200577 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287076+0000", 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014773100000000001 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.680466+0000", 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39339001800000001 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.680567+0000", 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100756 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.892064+0000", 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.211497712 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:440 3.6s0 3:674cc614:::smithi17829308-50:head [create,setxattr pnc]aaY (11788) in=7b,setxattr ZTVVStUpk__YaS\\blR\\XgesYT[Or[[aUett_nil] (1859) in=40b,setxattr ZtotWOVcPP_oiTUUTlqtYZsNneQnmhc\\bNdNpsgo (1402) in=40b,setxattr cR\\akUitgYfao[oisZ^ach\\i_kdge\\[c_mbmUf]^ (313) in=40b,setxattr pRe[UcWhXR^oagaSYScYPQa]r[hdldO\\S`V^[nh^ (874) in=40b,setxattr srVXOoW^mbi\\ccUpc`barbsc (11788) in=24b,setxattr pqjhUkafUjRqdpObgi_ZX^QV (11788) in=24b,setxattr q]Of\\bd_VgdoaP\\k^tmmpUqq^otrVVOdrtgU]nVk (12280) in=40b,setxattr ]Ple__\\nrXPQYecmQXf_o^bgN`mofssSPQrYslSO (1859) in=40b,setxattr ZiQcjT`UZQ]es__dmo^h]NdV^`qT_hniQtVY`WsT (770) in=40b,setxattr gpT^oWmeYZpbVdojjWnUWO[YfZptiQhdWdhs]Zo^ (841) in=40b,setxattr `Zgeqf\\Rb][aUdTVXYhj[ealjcUnRaiY (11788) in=32b,setxattr [dN]aooYVPYS^Q`_`bqSNVSS^`ZT\\csYZSrnjtUT (11296) in=40b,setxattr ehcNktRfRjqWWi[^XWcZokN]Wb[YnkeNl\\lkt_t\\ (1262) in=40b,setxattr OdsZcnit[de]qNV (11788) in=15b,setxattr ^]^lenZo\\TjoZOpONRno]jtaWOd`tqNdWcYccWNk (560) in=40b,setxattr ZUohXUqoPmYaaTceWo\\SRigth\\qP[Tg_^^asjknX (1085) in=40b,setxattr lR`dUarlc[n`ctj^TlY\\QpagN_doPfV`lZQ^kNtY (526) in=40b,setxattr [iXNTskqfmN\\qVhqrm`rdXpaN_VaeRbd^O\\Z_gcY (1261) in=40b,setxattr t_jR_akNV_aXVR^icaga]cP]baNne_f_UdsoeVUk (1894) in=40b] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:46.229654+0000", 2024-04-26T04:33:08.602 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.339423504999999, 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.51173439300000001, 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 440 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.229654+0000", 2024-04-26T04:33:08.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.229654+0000", 2024-04-26T04:33:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.229658+0000", 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0999999999999997e-06 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.229850+0000", 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000191838 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.229897+0000", 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6409999999999998e-05 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.330001+0000", 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100104415 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.330173+0000", 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017236399999999999 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.330497+0000", 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032375699999999998 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.331849+0000", 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001352294 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.336320+0000", 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0044704990000000002 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.741060+0000", 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.40474016499999999 2024-04-26T04:33:08.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.741389+0000", 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032855100000000001 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:640 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~638976] snapc f=[f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:57.184457+0000", 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 11.384620796, 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.51936972400000003, 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 640 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.184457+0000", 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.184457+0000", 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.184460+0000", 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8899999999999999e-06 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.185893+0000", 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001433141 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.185929+0000", 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6316999999999998e-05 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.185946+0000", 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7085999999999999e-05 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286217+0000", 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10027051400000001 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op must wait for map", 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286230+0000", 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2913e-05 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.390036+0000", 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.103806092 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.390304+0000", 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026764699999999998 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.393740+0000", 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0034366779999999999 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.602743+0000", 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20900233100000001 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.703687+0000", 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100944345 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.703827+0000", 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013977 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:641 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~458752] snapc f=[f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:57.188152+0000", 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 11.380925620999999, 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.516167932, 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.614 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 641 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.188152+0000", 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.188152+0000", 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.188154+0000", 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2239999999999998e-06 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.188810+0000", 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00065584599999999997 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.188843+0000", 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2437000000000001e-05 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.188858+0000", 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.571e-05 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286333+0000", 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.097474855999999999 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286345+0000", 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1353e-05 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.395969+0000", 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.109624634 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.396150+0000", 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000180674 2024-04-26T04:33:08.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.397825+0000", 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001675097 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.602815+0000", 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.204989807 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.704208+0000", 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101393371 2024-04-26T04:33:08.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.704320+0000", 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011192300000000001 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:642 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~704512] snapc f=[f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:57.193975+0000", 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 11.375102419999999, 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.51105824799999999, 2024-04-26T04:33:08.620 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 642 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.193975+0000", 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.193975+0000", 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.193978+0000", 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1569999999999998e-06 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.194951+0000", 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00097345999999999999 2024-04-26T04:33:08.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.194983+0000", 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1603000000000002e-05 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.194998+0000", 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5586999999999998e-05 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286444+0000", 2024-04-26T04:33:08.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.091445968000000002 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286455+0000", 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1250000000000001e-05 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.499305+0000", 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.21284921200000001 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.499480+0000", 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000175434 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.501311+0000", 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018308199999999999 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.602818+0000", 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101507166 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.704928+0000", 2024-04-26T04:33:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.102110311 2024-04-26T04:33:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.705034+0000", 2024-04-26T04:33:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010527999999999999 2024-04-26T04:33:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:643 3.6s0 3:647b920d:::smithi17829308-33:head [setxattr _header (57) in=7b] snapc f=[f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:33:08.626 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:57.195028+0000", 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 11.374050236, 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.81557969699999999, 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 643 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195028+0000", 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195028+0000", 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195029+0000", 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.06e-06 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195034+0000", 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6829999999999996e-06 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195050+0000", 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6056999999999999e-05 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195059+0000", 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0499999999999997e-06 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286572+0000", 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.091513091000000005 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286596+0000", 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3617000000000001e-05 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.705883+0000", 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.41928652700000002 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.706176+0000", 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029326599999999998 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.707464+0000", 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001287727 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.809419+0000", 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101955776 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:58.010340+0000", 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20092080400000001 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:58.010607+0000", 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000267039 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:644 3.6s0 3:647b920d:::smithi17829308-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:33:08.632 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:57.195074+0000", 2024-04-26T04:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 11.374004003, 2024-04-26T04:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.217959426, 2024-04-26T04:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 644 2024-04-26T04:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195074+0000", 2024-04-26T04:33:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195074+0000", 2024-04-26T04:33:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195075+0000", 2024-04-26T04:33:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3300000000000001e-07 2024-04-26T04:33:08.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195078+0000", 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6669999999999998e-06 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195093+0000", 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4413000000000001e-05 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195101+0000", 2024-04-26T04:33:08.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6000000000000007e-06 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286649+0000", 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.091547484999999998 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286657+0000", 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7900000000000005e-06 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.708272+0000", 2024-04-26T04:33:08.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.42161408700000003 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.708548+0000", 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027624000000000003 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:58.011546+0000", 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.30299795800000001 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:58.011801+0000", 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025481000000000001 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:58.413033+0000", 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.401232643 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:749 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~524288] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.399186+0000", 2024-04-26T04:33:08.638 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 6.1698918410000001, 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50750576999999997, 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 749 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.399186+0000", 2024-04-26T04:33:08.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.399186+0000", 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.399190+0000", 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0500000000000002e-06 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.400238+0000", 2024-04-26T04:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00104769 2024-04-26T04:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.400274+0000", 2024-04-26T04:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5830000000000001e-05 2024-04-26T04:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.400298+0000", 2024-04-26T04:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.474e-05 2024-04-26T04:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.400479+0000", 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018065999999999999 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.401431+0000", 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00095188300000000005 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404962+0000", 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0035312270000000001 2024-04-26T04:33:08.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.416625+0000", 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011662522 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.906553+0000", 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.48992856499999998 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.906692+0000", 2024-04-26T04:33:08.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138603 2024-04-26T04:33:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:750 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~573440] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.403748+0000", 2024-04-26T04:33:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 6.1653293639999998, 2024-04-26T04:33:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50340580899999998, 2024-04-26T04:33:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.644 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 750 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.403748+0000", 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.403748+0000", 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.403752+0000", 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4240000000000002e-06 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404702+0000", 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00095022599999999998 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404728+0000", 2024-04-26T04:33:08.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5530000000000001e-05 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404742+0000", 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4867e-05 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.408057+0000", 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0033145869999999999 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.408241+0000", 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000184303 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.409896+0000", 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0016550359999999999 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.416851+0000", 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0069544619999999998 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907022+0000", 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49017117799999999 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907154+0000", 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013219600000000001 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:751 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~688128] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.408886+0000", 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 6.1601914530000004, 2024-04-26T04:33:08.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49855173800000002, 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 751 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.408886+0000", 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.408886+0000", 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.408889+0000", 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5830000000000001e-06 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.410008+0000", 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001119292 2024-04-26T04:33:08.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.410038+0000", 2024-04-26T04:33:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9714e-05 2024-04-26T04:33:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.410056+0000", 2024-04-26T04:33:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8490000000000001e-05 2024-04-26T04:33:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411114+0000", 2024-04-26T04:33:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001057842 2024-04-26T04:33:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411263+0000", 2024-04-26T04:33:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014863699999999999 2024-04-26T04:33:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.413187+0000", 2024-04-26T04:33:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0019243719999999999 2024-04-26T04:33:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.416854+0000", 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0036668899999999999 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907328+0000", 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.490473822 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907438+0000", 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110096 2024-04-26T04:33:08.654 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:752 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~294912] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.410845+0000", 2024-04-26T04:33:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 6.1582327020000003, 2024-04-26T04:33:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49685625700000002, 2024-04-26T04:33:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.655 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 752 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.410845+0000", 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.410845+0000", 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.410847+0000", 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8870000000000001e-06 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411302+0000", 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00045527600000000001 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411327+0000", 2024-04-26T04:33:08.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4600000000000002e-05 2024-04-26T04:33:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411341+0000", 2024-04-26T04:33:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4100000000000001e-05 2024-04-26T04:33:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.414402+0000", 2024-04-26T04:33:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0030614750000000001 2024-04-26T04:33:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.414568+0000", 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016537999999999999 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.415660+0000", 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010918620000000001 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.418568+0000", 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0029087280000000002 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907598+0000", 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.48902954599999998 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907701+0000", 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000103403 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.660 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:753 3.6s0 3:647b920d:::smithi17829308-33:head [setxattr _header (57) in=7b] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.411373+0000", 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 6.1577048650000004, 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.496728699, 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 753 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411373+0000", 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411373+0000", 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411374+0000", 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1629999999999999e-06 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411379+0000", 2024-04-26T04:33:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1370000000000001e-06 2024-04-26T04:33:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411396+0000", 2024-04-26T04:33:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6330000000000001e-05 2024-04-26T04:33:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411405+0000", 2024-04-26T04:33:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0599999999999997e-06 2024-04-26T04:33:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.416890+0000", 2024-04-26T04:33:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.005485696 2024-04-26T04:33:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.417027+0000", 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000137026 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.417828+0000", 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00080056599999999995 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.419274+0000", 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001446396 2024-04-26T04:33:08.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907952+0000", 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.48867820200000001 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.908102+0000", 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000149123 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.667 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:754 3.6s0 3:647b920d:::smithi17829308-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.411418+0000", 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 6.1576593089999996, 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.60016049599999999, 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 754 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411418+0000", 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411418+0000", 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411419+0000", 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4000000000000001e-07 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411423+0000", 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4400000000000001e-06 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411434+0000", 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1707e-05 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411443+0000", 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9129999999999992e-06 2024-04-26T04:33:08.670 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.418404+0000", 2024-04-26T04:33:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0069603850000000004 2024-04-26T04:33:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.418547+0000", 2024-04-26T04:33:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000143537 2024-04-26T04:33:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.671 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.908913+0000", 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49036608399999998 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.909197+0000", 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028418999999999997 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.011579+0000", 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.1023815 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.672 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 68/30 ECSubWriteReply(tid=66, last_complete=68'47, committed=1, applied=1))", 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:03.393865+0000", 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.1752128969999998, 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.42230022499999997, 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.393865+0000", 2024-04-26T04:33:08.673 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.393865+0000", 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.393867+0000", 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.714e-06 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.674 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.393876+0000", 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6059999999999992e-06 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.393905+0000", 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9244e-05 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.494075+0000", 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10016963800000001 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.675 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.816132+0000", 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.32205738 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.816165+0000", 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2642999999999999e-05 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 68/30 ECSubWriteReply(tid=66, last_complete=68'47, committed=1, applied=1))", 2024-04-26T04:33:08.676 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:03.518321+0000", 2024-04-26T04:33:08.677 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0507564870000001, 2024-04-26T04:33:08.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39861305200000002, 2024-04-26T04:33:08.677 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.677 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.677 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:33:08.677 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.677 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518321+0000", 2024-04-26T04:33:08.677 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.677 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.677 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.677 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518321+0000", 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518323+0000", 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.393e-06 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518332+0000", 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6039999999999999e-06 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.678 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518368+0000", 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5269999999999999e-05 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518388+0000", 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.031e-05 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.916728+0000", 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39833988199999998 2024-04-26T04:33:08.679 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.916934+0000", 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020659300000000001 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 68/30 ECSubWriteReply(tid=67, last_complete=68'49, committed=1, applied=1))", 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:03.518571+0000", 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 5.0505070999999999, 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.59897118599999999, 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:08.680 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518571+0000", 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518571+0000", 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.681 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518572+0000", 2024-04-26T04:33:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1429999999999999e-06 2024-04-26T04:33:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518579+0000", 2024-04-26T04:33:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6529999999999994e-06 2024-04-26T04:33:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518601+0000", 2024-04-26T04:33:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1474000000000001e-05 2024-04-26T04:33:08.682 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518613+0000", 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:04.116715+0000", 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.59810200499999999 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:04.117542+0000", 2024-04-26T04:33:08.683 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00082731099999999995 2024-04-26T04:33:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:08.684 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:08.684 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:08.685 INFO:tasks.rados.rados.0.smithi178.stdout:317: done (1 left) 2024-04-26T04:33:08.685 INFO:tasks.rados.rados.0.smithi178.stdout:322: done (0 left) 2024-04-26T04:33:08.686 INFO:tasks.rados.rados.0.smithi178.stdout:323: rollback oid 18 current snap is 20 2024-04-26T04:33:08.686 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 18 to 15 2024-04-26T04:33:08.686 INFO:tasks.rados.rados.0.smithi178.stdout:324: snap_create 2024-04-26T04:33:08.686 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T04:33:08.687 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:08.682+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:08.786 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:08.782+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:08.819 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:33:08.819 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-04-26T04:33:08.819 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:08.833 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T04:33:08.887 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:08.885+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:08.960 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:08.960 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:33:08.960 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-04-26T04:33:08.960 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-04-26T04:33:08.960 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:08.975 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T04:33:08.988 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:08.984+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:09.089 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:09.089+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:09.099 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-04-26T04:33:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-04-26T04:33:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:33:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.099 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:236 3.1s0 3:8f617591:::smithi17829308-17:head [delete] snapc 2=[2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e35)", 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:28.808956+0000", 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 40.286021806000001, 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.92629017999999996, 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 236 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.100 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.808956+0000", 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.808956+0000", 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.808958+0000", 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9800000000000001e-06 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.101 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.808967+0000", 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8230000000000006e-06 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.808987+0000", 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9664999999999999e-05 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.808998+0000", 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0740999999999999e-05 2024-04-26T04:33:09.102 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.630849+0000", 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.821851424 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.631005+0000", 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000155818 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.632065+0000", 2024-04-26T04:33:09.103 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010597359999999999 2024-04-26T04:33:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.633919+0000", 2024-04-26T04:33:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018537479999999999 2024-04-26T04:33:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.104 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.735099+0000", 2024-04-26T04:33:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101180744 2024-04-26T04:33:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.735246+0000", 2024-04-26T04:33:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000146501 2024-04-26T04:33:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.105 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:240 3.1s0 3:83946740:::smithi17829308-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:29.766578+0000", 2024-04-26T04:33:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 39.328399437999998, 2024-04-26T04:33:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.795088193, 2024-04-26T04:33:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.106 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 240 2024-04-26T04:33:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.766578+0000", 2024-04-26T04:33:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.107 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.766578+0000", 2024-04-26T04:33:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.766579+0000", 2024-04-26T04:33:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6300000000000004e-07 2024-04-26T04:33:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.766582+0000", 2024-04-26T04:33:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3699999999999999e-06 2024-04-26T04:33:09.108 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.766612+0000", 2024-04-26T04:33:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9963000000000001e-05 2024-04-26T04:33:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.766624+0000", 2024-04-26T04:33:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1557e-05 2024-04-26T04:33:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.109 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.048520+0000", 2024-04-26T04:33:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.28189626099999998 2024-04-26T04:33:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.048753+0000", 2024-04-26T04:33:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000232699 2024-04-26T04:33:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.267779+0000", 2024-04-26T04:33:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.219026107 2024-04-26T04:33:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.110 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.268049+0000", 2024-04-26T04:33:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027038700000000003 2024-04-26T04:33:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.561666+0000", 2024-04-26T04:33:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.29361708600000003 2024-04-26T04:33:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.111 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:251 3.1s0 3:8f617591:::smithi17829308-17:head [copy-from ver 5] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:29.790926+0000", 2024-04-26T04:33:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 39.304051688000001, 2024-04-26T04:33:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.97147092499999999, 2024-04-26T04:33:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 251 2024-04-26T04:33:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.112 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.790926+0000", 2024-04-26T04:33:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.790926+0000", 2024-04-26T04:33:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.113 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.790928+0000", 2024-04-26T04:33:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2139999999999999e-06 2024-04-26T04:33:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.790940+0000", 2024-04-26T04:33:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1542999999999999e-05 2024-04-26T04:33:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.790970+0000", 2024-04-26T04:33:09.114 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0574000000000002e-05 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.790985+0000", 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5299999999999999e-05 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.048921+0000", 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.25793591700000001 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.115 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.049220+0000", 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029858 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.375684+0000", 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.32646407300000002 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.377851+0000", 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002167141 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.116 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.762292+0000", 2024-04-26T04:33:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.38444051499999998 2024-04-26T04:33:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.762397+0000", 2024-04-26T04:33:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000105068 2024-04-26T04:33:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.117 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:258 3.1s0 3:8c067cfd:::smithi17829308-21:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:29.892806+0000", 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 39.202171638000003, 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0716308809999999, 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 258 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.118 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.892806+0000", 2024-04-26T04:33:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.892806+0000", 2024-04-26T04:33:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.119 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.892807+0000", 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1999999999999999e-07 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.892810+0000", 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0939999999999999e-06 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.892820+0000", 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0236e-05 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.120 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.892828+0000", 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7670000000000003e-06 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.057801+0000", 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.16497324199999999 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.058058+0000", 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000256879 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.121 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.560399+0000", 2024-04-26T04:33:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50234152799999998 2024-04-26T04:33:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.560550+0000", 2024-04-26T04:33:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015108100000000001 2024-04-26T04:33:09.122 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.964437+0000", 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.40388633400000001 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:260 3.1s0 3:8bd869b7:::smithi17829308-2:head [append 0~376832] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:29.993865+0000", 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 39.101112851000003, 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.76692751000000003, 2024-04-26T04:33:09.123 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 260 2024-04-26T04:33:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.124 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.993865+0000", 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.993865+0000", 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.993866+0000", 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6330000000000001e-06 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.125 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994355+0000", 2024-04-26T04:33:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00048898300000000002 2024-04-26T04:33:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994380+0000", 2024-04-26T04:33:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4507000000000001e-05 2024-04-26T04:33:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994392+0000", 2024-04-26T04:33:09.126 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2496e-05 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.161949+0000", 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.16755651699999999 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.162219+0000", 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026998000000000001 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.164138+0000", 2024-04-26T04:33:09.127 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001919125 2024-04-26T04:33:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.266155+0000", 2024-04-26T04:33:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.102017224 2024-04-26T04:33:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.760676+0000", 2024-04-26T04:33:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49452143999999998 2024-04-26T04:33:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.128 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.760792+0000", 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000115605 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:261 3.1s0 3:8bd869b7:::smithi17829308-2:head [setxattr _header (55) in=7b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:29.994409+0000", 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 39.100568627999998, 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.76693917899999997, 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:09.129 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 261 2024-04-26T04:33:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994409+0000", 2024-04-26T04:33:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.130 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994409+0000", 2024-04-26T04:33:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994410+0000", 2024-04-26T04:33:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8599999999999996e-07 2024-04-26T04:33:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994419+0000", 2024-04-26T04:33:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.037e-06 2024-04-26T04:33:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.131 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994430+0000", 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0794e-05 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994438+0000", 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9759999999999995e-06 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.370941+0000", 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.376503531 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.132 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.371218+0000", 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027671299999999999 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.372480+0000", 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001261664 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.377847+0000", 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0053671650000000001 2024-04-26T04:33:09.133 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.761216+0000", 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.38336967199999999 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.761348+0000", 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000131641 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.134 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:338 3.7s0 3:e1f05b66:::smithi17829308-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.147909+0000", 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 33.947068725999998, 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.73236990000000002, 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 338 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.135 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.147909+0000", 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.147909+0000", 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.147910+0000", 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5239999999999999e-06 2024-04-26T04:33:09.136 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.147918+0000", 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3599999999999998e-06 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.147940+0000", 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.247e-05 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.147957+0000", 2024-04-26T04:33:09.137 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7374e-05 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.436335+0000", 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.28837752999999999 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.436482+0000", 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014667 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.138 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.644516+0000", 2024-04-26T04:33:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20803428900000001 2024-04-26T04:33:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.644756+0000", 2024-04-26T04:33:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023993199999999999 2024-04-26T04:33:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.880279+0000", 2024-04-26T04:33:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.235522751 2024-04-26T04:33:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.139 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:344 3.bs0 3:df125fb3:::smithi17829308-41:head [copy-from ver 6] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180126+0000", 2024-04-26T04:33:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 33.914851747999997, 2024-04-26T04:33:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.76992672399999995, 2024-04-26T04:33:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.140 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 344 2024-04-26T04:33:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180126+0000", 2024-04-26T04:33:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.141 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180126+0000", 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180130+0000", 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0969999999999999e-06 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180148+0000", 2024-04-26T04:33:09.142 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7880000000000002e-05 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180183+0000", 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5612999999999999e-05 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180200+0000", 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6529999999999999e-05 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.143 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180372+0000", 2024-04-26T04:33:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017194500000000001 2024-04-26T04:33:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180532+0000", 2024-04-26T04:33:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016060400000000001 2024-04-26T04:33:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.647255+0000", 2024-04-26T04:33:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.46672253299999999 2024-04-26T04:33:09.144 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.649209+0000", 2024-04-26T04:33:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0019545180000000001 2024-04-26T04:33:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.949989+0000", 2024-04-26T04:33:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.300779095 2024-04-26T04:33:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.145 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.950052+0000", 2024-04-26T04:33:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3909000000000001e-05 2024-04-26T04:33:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:358 3.1s0 3:8f617591:::smithi17829308-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:33:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.339814+0000", 2024-04-26T04:33:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 33.755163541999998, 2024-04-26T04:33:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.112259364, 2024-04-26T04:33:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.146 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 358 2024-04-26T04:33:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339814+0000", 2024-04-26T04:33:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.147 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339814+0000", 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339815+0000", 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2300000000000001e-07 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339817+0000", 2024-04-26T04:33:09.148 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8729999999999999e-06 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339827+0000", 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2669999999999995e-06 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339834+0000", 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0539999999999997e-06 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.149 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339863+0000", 2024-04-26T04:33:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.904e-05 2024-04-26T04:33:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339867+0000", 2024-04-26T04:33:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9779999999999999e-06 2024-04-26T04:33:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053012+0000", 2024-04-26T04:33:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.71314480999999996 2024-04-26T04:33:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.150 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053102+0000", 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0791999999999994e-05 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.452073+0000", 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39897092699999998 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.151 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:373 3.7s0 3:e1f05b66:::smithi17829308-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.411932+0000", 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 29.683045001, 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.80901803500000002, 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 373 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.152 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.411932+0000", 2024-04-26T04:33:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.411932+0000", 2024-04-26T04:33:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.411933+0000", 2024-04-26T04:33:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9700000000000003e-07 2024-04-26T04:33:09.153 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.411934+0000", 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6169999999999999e-06 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.411940+0000", 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7239999999999997e-06 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.512013+0000", 2024-04-26T04:33:09.154 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10007334900000001 2024-04-26T04:33:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.515655+0000", 2024-04-26T04:33:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0036413909999999999 2024-04-26T04:33:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.515734+0000", 2024-04-26T04:33:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9039e-05 2024-04-26T04:33:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.155 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.919808+0000", 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.40407443100000001 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.919954+0000", 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000146081 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.220950+0000", 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.30099610599999999 2024-04-26T04:33:09.156 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:448 3.1s0 3:8bd869b7:::smithi17829308-2:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:46.262520+0000", 2024-04-26T04:33:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.832457009999999, 2024-04-26T04:33:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.716085101, 2024-04-26T04:33:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:09.157 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 448 2024-04-26T04:33:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.262520+0000", 2024-04-26T04:33:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.158 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.262520+0000", 2024-04-26T04:33:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.262521+0000", 2024-04-26T04:33:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4499999999999997e-07 2024-04-26T04:33:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.262523+0000", 2024-04-26T04:33:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.407e-06 2024-04-26T04:33:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.262531+0000", 2024-04-26T04:33:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5100000000000001e-06 2024-04-26T04:33:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.262535+0000", 2024-04-26T04:33:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6380000000000003e-06 2024-04-26T04:33:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.572760+0000", 2024-04-26T04:33:09.160 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.31022430499999998 2024-04-26T04:33:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.572940+0000", 2024-04-26T04:33:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000180667 2024-04-26T04:33:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.777913+0000", 2024-04-26T04:33:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20497271 2024-04-26T04:33:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.778070+0000", 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015688 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.978606+0000", 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20053553900000001 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:600 3.7s0 3:ee767cea:::smithi17829308-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:09.162 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:55.877365+0000", 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.217612547, 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1179588549999999, 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 600 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877365+0000", 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877365+0000", 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877367+0000", 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9599999999999999e-06 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877378+0000", 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1043000000000001e-05 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877401+0000", 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2974000000000001e-05 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877414+0000", 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.279e-05 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.085975+0000", 2024-04-26T04:33:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.208561152 2024-04-26T04:33:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.086223+0000", 2024-04-26T04:33:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024830400000000003 2024-04-26T04:33:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.589716+0000", 2024-04-26T04:33:09.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50349285499999996 2024-04-26T04:33:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.589840+0000", 2024-04-26T04:33:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012359 2024-04-26T04:33:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.995324+0000", 2024-04-26T04:33:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.405484187 2024-04-26T04:33:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.167 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:601 3.7s0 3:e1054bf1:::smithi17829308-37:head [read 0~606208,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:55.877432+0000", 2024-04-26T04:33:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.217545724000001, 2024-04-26T04:33:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.81405845700000001, 2024-04-26T04:33:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.168 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 601 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877432+0000", 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877432+0000", 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877433+0000", 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6000000000000002e-07 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877438+0000", 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.784e-06 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877452+0000", 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4056000000000001e-05 2024-04-26T04:33:09.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877461+0000", 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5739999999999996e-06 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.086318+0000", 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20885710599999999 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.086752+0000", 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00043374 2024-04-26T04:33:09.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.691490+0000", 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.60473833700000001 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:602 3.7s0 3:e1054bf1:::smithi17829308-37:head [read 0~606208,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:55.877473+0000", 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.2175049, 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.81550703000000002, 2024-04-26T04:33:09.172 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 602 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877473+0000", 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.173 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877473+0000", 2024-04-26T04:33:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877473+0000", 2024-04-26T04:33:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6599999999999995e-07 2024-04-26T04:33:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877479+0000", 2024-04-26T04:33:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9939999999999997e-06 2024-04-26T04:33:09.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877492+0000", 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2353e-05 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877499+0000", 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4399999999999999e-06 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.087707+0000", 2024-04-26T04:33:09.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.21020797299999999 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.087955+0000", 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024785699999999998 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.692980+0000", 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.60502464700000003 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:603 3.7s0 3:e1054bf1:::smithi17829308-37:head [read 0~606208,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:55.877509+0000", 2024-04-26T04:33:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 13.217468917, 2024-04-26T04:33:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0172811070000001, 2024-04-26T04:33:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 603 2024-04-26T04:33:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.177 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877509+0000", 2024-04-26T04:33:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877509+0000", 2024-04-26T04:33:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.178 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877509+0000", 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6700000000000003e-07 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877513+0000", 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7859999999999998e-06 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877525+0000", 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.205e-05 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.977632+0000", 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100106602 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.089944+0000", 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.112312382 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.090218+0000", 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027392999999999997 2024-04-26T04:33:09.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.894790+0000", 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.80457159 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:709 3.bs0 3:d4731d0d:::smithi17829308-42:head [read 0~638976,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:01.680178+0000", 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 7.414799661, 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.71605977600000004, 2024-04-26T04:33:09.181 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 709 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.680178+0000", 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.680178+0000", 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.680178+0000", 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9200000000000001e-07 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.680184+0000", 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.8680000000000004e-06 2024-04-26T04:33:09.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.680192+0000", 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2730000000000002e-06 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.680199+0000", 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0990000000000004e-06 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.682314+0000", 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002115699 2024-04-26T04:33:09.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.682411+0000", 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6459999999999995e-05 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.396238+0000", 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.71382688500000002 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.185 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:720 3.bs0 3:d909d6e5:::smithi17829308-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e66)", 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:01.682079+0000", 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 7.4128984320000004, 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.92979241599999995, 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 720 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.682079+0000", 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.682079+0000", 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.682079+0000", 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1199999999999999e-07 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.682081+0000", 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3740000000000001e-06 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.682085+0000", 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0609999999999997e-06 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.782170+0000", 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100084801 2024-04-26T04:33:09.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.396384+0000", 2024-04-26T04:33:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.61421400199999998 2024-04-26T04:33:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.396662+0000", 2024-04-26T04:33:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027891 2024-04-26T04:33:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.611871+0000", 2024-04-26T04:33:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.21520895600000001 2024-04-26T04:33:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:814 3.bs0 3:d909d6e5:::smithi17829308-49:head [append 0~0] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:07.275388+0000", 2024-04-26T04:33:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.819589342, 2024-04-26T04:33:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.75181568200000004, 2024-04-26T04:33:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.190 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 814 2024-04-26T04:33:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275388+0000", 2024-04-26T04:33:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275388+0000", 2024-04-26T04:33:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275390+0000", 2024-04-26T04:33:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2180000000000002e-06 2024-04-26T04:33:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275399+0000", 2024-04-26T04:33:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7409999999999998e-06 2024-04-26T04:33:09.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275421+0000", 2024-04-26T04:33:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1926999999999999e-05 2024-04-26T04:33:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275434+0000", 2024-04-26T04:33:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3355e-05 2024-04-26T04:33:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.415905+0000", 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.14047053800000001 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op must wait for map", 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.415910+0000", 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.395e-06 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.516958+0000", 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10104777299999999 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.517130+0000", 2024-04-26T04:33:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017210000000000001 2024-04-26T04:33:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.518464+0000", 2024-04-26T04:33:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0013335510000000001 2024-04-26T04:33:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.521142+0000", 2024-04-26T04:33:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0026784700000000001 2024-04-26T04:33:09.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.027069+0000", 2024-04-26T04:33:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50592662799999999 2024-04-26T04:33:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.027204+0000", 2024-04-26T04:33:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134986 2024-04-26T04:33:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:815 3.bs0 3:d909d6e5:::smithi17829308-49:head [rollback 14] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:07.275463+0000", 2024-04-26T04:33:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.81951426, 2024-04-26T04:33:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.85294690699999998, 2024-04-26T04:33:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 815 2024-04-26T04:33:09.197 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275463+0000", 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275463+0000", 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.198 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275464+0000", 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0900000000000001e-07 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275467+0000", 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0819999999999999e-06 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275474+0000", 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6619999999999999e-06 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.199 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275480+0000", 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3820000000000001e-06 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.724313+0000", 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.44883297900000002 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.724463+0000", 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014975000000000001 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.725424+0000", 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00096111600000000003 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.127662+0000", 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.40223767500000002 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.128209+0000", 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00054786899999999998 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.128410+0000", 2024-04-26T04:33:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000200683 2024-04-26T04:33:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:816 3.bs0 3:d909d6e5:::smithi17829308-49:head [append 0~0] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:07.275523+0000", 2024-04-26T04:33:09.202 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.8194543489999999, 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0583476979999999, 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 816 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275523+0000", 2024-04-26T04:33:09.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275523+0000", 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275524+0000", 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8500000000000002e-07 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:09.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275526+0000", 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4600000000000002e-06 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275532+0000", 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1260000000000003e-06 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275538+0000", 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0260000000000004e-06 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.205 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.230416+0000", 2024-04-26T04:33:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.954877592 2024-04-26T04:33:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.230595+0000", 2024-04-26T04:33:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017954700000000001 2024-04-26T04:33:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.231448+0000", 2024-04-26T04:33:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00085291700000000002 2024-04-26T04:33:09.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.232431+0000", 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00098273900000000005 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.333824+0000", 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101392836 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.333871+0000", 2024-04-26T04:33:09.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6969999999999999e-05 2024-04-26T04:33:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:09.208 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:09.208 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:09.210 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T04:33:09.210 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T04:33:09.211 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:09.206+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:09.268 INFO:tasks.rados.rados.0.smithi178.stdout:324: done (1 left) 2024-04-26T04:33:09.268 INFO:tasks.rados.rados.0.smithi178.stdout:325: rollback oid 9 current snap is 21 2024-04-26T04:33:09.268 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 9 to 16 2024-04-26T04:33:09.268 INFO:tasks.rados.rados.0.smithi178.stdout:326: rollback oid 14 current snap is 21 2024-04-26T04:33:09.268 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 14 to 18 2024-04-26T04:33:09.268 INFO:tasks.rados.rados.0.smithi178.stdout:327: rmattr oid 30 current snap is 21 2024-04-26T04:33:09.268 INFO:tasks.rados.rados.0.smithi178.stdout:327: done (3 left) 2024-04-26T04:33:09.268 INFO:tasks.rados.rados.0.smithi178.stdout:328: copy_from oid 13 from oid 37 current snap is 21 2024-04-26T04:33:09.269 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 36 current snap is 21 2024-04-26T04:33:09.269 INFO:tasks.rados.rados.0.smithi178.stdout:329: seq_num 132 ranges {589824=671744,1261568=573440,1835008=753664,2588672=229376} 2024-04-26T04:33:09.289 INFO:tasks.rados.rados.0.smithi178.stdout:329: writing smithi17829308-36 from 589824 to 1261568 tid 1 2024-04-26T04:33:09.299 INFO:tasks.rados.rados.0.smithi178.stdout:329: writing smithi17829308-36 from 1261568 to 1835008 tid 2 2024-04-26T04:33:09.310 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:09.309+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:09.311 INFO:tasks.rados.rados.0.smithi178.stdout:329: writing smithi17829308-36 from 1835008 to 2588672 tid 3 2024-04-26T04:33:09.315 INFO:tasks.rados.rados.0.smithi178.stdout:329: writing smithi17829308-36 from 2588672 to 2818048 tid 4 2024-04-26T04:33:09.315 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 31 current snap is 21 2024-04-26T04:33:09.315 INFO:tasks.rados.rados.0.smithi178.stdout:330: seq_num 133 ranges {524288=688128,1212416=720896,1933312=475136,2408448=360448} 2024-04-26T04:33:09.330 INFO:tasks.rados.rados.0.smithi178.stdout:330: writing smithi17829308-31 from 524288 to 1212416 tid 1 2024-04-26T04:33:09.338 INFO:tasks.rados.rados.0.smithi178.stdout:330: writing smithi17829308-31 from 1212416 to 1933312 tid 2 2024-04-26T04:33:09.343 INFO:tasks.rados.rados.0.smithi178.stdout:330: writing smithi17829308-31 from 1933312 to 2408448 tid 3 2024-04-26T04:33:09.345 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:33:09.345 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-04-26T04:33:09.345 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:09.346 INFO:tasks.rados.rados.0.smithi178.stdout:330: writing smithi17829308-31 from 2408448 to 2768896 tid 4 2024-04-26T04:33:09.346 INFO:tasks.rados.rados.0.smithi178.stdout:331: setattr oid 39 current snap is 21 2024-04-26T04:33:09.348 INFO:tasks.rados.rados.0.smithi178.stdout:332: snap_create 2024-04-26T04:33:09.361 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T04:33:09.411 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:09.410+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:09.412 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:33:09.412 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:33:09.416 INFO:tasks.rados.rados.0.smithi178.stdout:329: finishing write tid 1 to smithi17829308-36 2024-04-26T04:33:09.435 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:33:09.436 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:33:09.462 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:33:09.462 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:33:09.481 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:33:09.481 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-04-26T04:33:09.482 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-04-26T04:33:09.482 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:09.487 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:33:09.487 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:33:09.496 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T04:33:09.508 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:33:09.509 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:33:09.512 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:09.509+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:09.532 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:33:09.533 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:33:09.560 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:33:09.560 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:33:09.580 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:33:09.581 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:33:09.597 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:33:09.597 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:33:09.613 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:09.608+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:09.620 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:09.621 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:09.636 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T04:33:09.645 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 39 v 63 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-26T04:33:09.678 INFO:tasks.rados.rados.0.smithi178.stdout:325: finishing rollback tid 0 to smithi17829308-9 2024-04-26T04:33:09.678 INFO:tasks.rados.rados.0.smithi178.stdout:325: finishing rollback tid 1 to smithi17829308-9 2024-04-26T04:33:09.714 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:09.712+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:09.780 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:33:09.781 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:33:09.815 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:33:09.810+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:33:09.816 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:33:09.817 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:33:09.822 INFO:tasks.rados.rados.0.smithi178.stdout:329: finishing write tid 2 to smithi17829308-36 2024-04-26T04:33:09.822 INFO:tasks.rados.rados.0.smithi178.stdout:329: finishing write tid 3 to smithi17829308-36 2024-04-26T04:33:09.822 INFO:tasks.rados.rados.0.smithi178.stdout:329: finishing write tid 4 to smithi17829308-36 2024-04-26T04:33:09.823 INFO:tasks.rados.rados.0.smithi178.stdout:329: finishing write tid 5 to smithi17829308-36 2024-04-26T04:33:09.833 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:09.833 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:33:09.834 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-04-26T04:33:09.834 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:09.846 INFO:tasks.rados.rados.0.smithi178.stdout:330: finishing write tid 1 to smithi17829308-31 2024-04-26T04:33:09.853 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T04:33:09.854 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:33:09.854 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:33:09.871 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:33:09.872 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:33:09.879 INFO:tasks.rados.rados.0.smithi178.stdout:325: finishing rollback tid 2 to smithi17829308-9 2024-04-26T04:33:09.879 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 9 v 81 (ObjNum 87 snap 12 seq_num 87) dirty exists 2024-04-26T04:33:09.894 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:33:09.894 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:33:09.915 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:33:09.915 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:33:09.915 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:09.913+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:09.926 INFO:tasks.rados.rados.0.smithi178.stdout:329: finishing write tid 6 to smithi17829308-36 2024-04-26T04:33:09.926 INFO:tasks.rados.rados.0.smithi178.stdout:329: oid 36 updating version 0 to 92 2024-04-26T04:33:09.926 INFO:tasks.rados.rados.0.smithi178.stdout:329: oid 36 version 92 is already newer than 89 2024-04-26T04:33:09.926 INFO:tasks.rados.rados.0.smithi178.stdout:329: oid 36 version 92 is already newer than 88 2024-04-26T04:33:09.926 INFO:tasks.rados.rados.0.smithi178.stdout:329: oid 36 version 92 is already newer than 90 2024-04-26T04:33:09.926 INFO:tasks.rados.rados.0.smithi178.stdout:329: oid 36 version 92 is already newer than 91 2024-04-26T04:33:09.926 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 36 v 92 (ObjNum 132 snap 21 seq_num 132) dirty exists 2024-04-26T04:33:09.926 INFO:tasks.rados.rados.0.smithi178.stdout:329: left oid 36 (ObjNum 132 snap 21 seq_num 132) 2024-04-26T04:33:09.936 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:33:09.937 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:33:09.946 INFO:tasks.rados.rados.0.smithi178.stdout:330: finishing write tid 2 to smithi17829308-31 2024-04-26T04:33:09.946 INFO:tasks.rados.rados.0.smithi178.stdout:330: finishing write tid 3 to smithi17829308-31 2024-04-26T04:33:09.955 INFO:tasks.rados.rados.0.smithi178.stdout:328: finishing copy_from racing read to smithi17829308-13 2024-04-26T04:33:09.977 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:09.977 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:09.992 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T04:33:10.016 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:10.013+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:10.056 INFO:tasks.rados.rados.0.smithi178.stdout:328: finishing copy_from to smithi17829308-13 2024-04-26T04:33:10.056 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 13 v 6 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-26T04:33:10.117 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:10.114+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:10.151 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:10.151 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:10.166 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T04:33:10.218 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:10.216+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:10.276 INFO:tasks.rados.rados.0.smithi178.stdout:325: done (7 left) 2024-04-26T04:33:10.277 INFO:tasks.rados.rados.0.smithi178.stdout:328: done (6 left) 2024-04-26T04:33:10.277 INFO:tasks.rados.rados.0.smithi178.stdout:329: done (5 left) 2024-04-26T04:33:10.277 INFO:tasks.rados.rados.0.smithi178.stdout:331: done (4 left) 2024-04-26T04:33:10.277 INFO:tasks.rados.rados.0.smithi178.stdout:332: done (3 left) 2024-04-26T04:33:10.277 INFO:tasks.rados.rados.0.smithi178.stdout:333: copy_from oid 22 from oid 28 current snap is 22 2024-04-26T04:33:10.277 INFO:tasks.rados.rados.0.smithi178.stdout:334: snap_remove snap 19 2024-04-26T04:33:10.279 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-26T04:33:10.299 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:10.299 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:10.313 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T04:33:10.361 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:10.361+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:10.420 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:10.417+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:10.436 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:10.436 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:10.450 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T04:33:10.521 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:10.516+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:10.613 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:10.613 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:10.622 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:10.618+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:10.627 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T04:33:10.722 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:10.720+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:10.750 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:10.750 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:10.764 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T04:33:10.824 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:10.824+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:10.885 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:10.885 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:10.899 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T04:33:10.925 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:10.920+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:11.016 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:11.017 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:11.025 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:11.025+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:11.031 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T04:33:11.126 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:11.124+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:11.170 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-04-26T04:33:11.176 INFO:tasks.rados.rados.0.smithi178.stdout:334: done (4 left) 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:335: rollback oid 32 current snap is 22 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 32 to 21 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:333: finishing copy_from to smithi17829308-22 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:333: got expected ENOENT (src dne) 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:333: finishing copy_from racing read to smithi17829308-22 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:330: finishing write tid 4 to smithi17829308-31 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:330: finishing write tid 5 to smithi17829308-31 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:330: finishing write tid 6 to smithi17829308-31 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:330: oid 31 updating version 0 to 60 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:330: oid 31 version 60 is already newer than 56 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:330: oid 31 version 60 is already newer than 57 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:330: oid 31 version 60 is already newer than 59 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:330: oid 31 version 60 is already newer than 58 2024-04-26T04:33:11.177 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 31 v 60 (ObjNum 133 snap 21 seq_num 133) dirty exists 2024-04-26T04:33:11.178 INFO:tasks.rados.rados.0.smithi178.stdout:330: left oid 31 (ObjNum 133 snap 21 seq_num 133) 2024-04-26T04:33:11.178 INFO:tasks.rados.rados.0.smithi178.stdout:330: done (4 left) 2024-04-26T04:33:11.178 INFO:tasks.rados.rados.0.smithi178.stdout:333: done (3 left) 2024-04-26T04:33:11.178 INFO:tasks.rados.rados.0.smithi178.stdout:336: read oid 22 snap -1 2024-04-26T04:33:11.178 INFO:tasks.rados.rados.0.smithi178.stdout:336: expect (ObjNum 21 snap 0 seq_num 21) 2024-04-26T04:33:11.178 INFO:tasks.rados.rados.0.smithi178.stdout:337: read oid 39 snap -1 2024-04-26T04:33:11.178 INFO:tasks.rados.rados.0.smithi178.stdout:337: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:33:11.178 INFO:tasks.rados.rados.0.smithi178.stdout:338: delete oid 19 current snap is 22 2024-04-26T04:33:11.198 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:11.198 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:11.212 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T04:33:11.227 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:11.226+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:11.328 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:11.325+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:11.342 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:11.343 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:11.358 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T04:33:11.391 INFO:tasks.rados.rados.0.smithi178.stdout:337: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:33:11.444 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:11.441+0000 7fc593d1c640 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 15 after trimming was started 2024-04-26T04:33:11.445 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:11.445+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:11.529 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:11.529 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:11.529 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:11.529+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:11.540 INFO:tasks.rados.rados.0.smithi178.stdout:337: done (5 left) 2024-04-26T04:33:11.540 INFO:tasks.rados.rados.0.smithi178.stdout:338: done (4 left) 2024-04-26T04:33:11.540 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 19 current snap is 22 2024-04-26T04:33:11.540 INFO:tasks.rados.rados.0.smithi178.stdout:339: seq_num 135 ranges {0=720896,720896=622592,1343488=688128,2031616=245760} 2024-04-26T04:33:11.544 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T04:33:11.549 INFO:tasks.rados.rados.0.smithi178.stdout:339: writing smithi17829308-19 from 0 to 720896 tid 1 2024-04-26T04:33:11.555 INFO:tasks.rados.rados.0.smithi178.stdout:339: writing smithi17829308-19 from 720896 to 1343488 tid 2 2024-04-26T04:33:11.563 INFO:tasks.rados.rados.0.smithi178.stdout:339: writing smithi17829308-19 from 1343488 to 2031616 tid 3 2024-04-26T04:33:11.565 INFO:tasks.rados.rados.0.smithi178.stdout:339: writing smithi17829308-19 from 2031616 to 2277376 tid 4 2024-04-26T04:33:11.566 INFO:tasks.rados.rados.0.smithi178.stdout:340: delete oid 35 current snap is 22 2024-04-26T04:33:11.630 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:11.626+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:11.638 INFO:tasks.rados.rados.0.smithi178.stdout:335: finishing rollback tid 0 to smithi17829308-32 2024-04-26T04:33:11.638 INFO:tasks.rados.rados.0.smithi178.stdout:335: finishing rollback tid 1 to smithi17829308-32 2024-04-26T04:33:11.720 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:11.720 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:11.731 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:11.730+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:11.734 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T04:33:11.738 INFO:tasks.rados.rados.0.smithi178.stdout:335: finishing rollback tid 2 to smithi17829308-32 2024-04-26T04:33:11.738 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 32 v 99 (ObjNum 87 snap 12 seq_num 87) dirty exists 2024-04-26T04:33:11.789 INFO:tasks.rados.rados.0.smithi178.stdout:336: expect (ObjNum 21 snap 0 seq_num 21) 2024-04-26T04:33:11.816 INFO:tasks.rados.rados.0.smithi178.stdout:323: finishing rollback tid 0 to smithi17829308-18 2024-04-26T04:33:11.832 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:11.830+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:11.853 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:11.853 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:11.867 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T04:33:11.907 INFO:tasks.rados.rados.0.smithi178.stdout:323: finishing rollback tid 1 to smithi17829308-18 2024-04-26T04:33:11.933 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:33:11.929+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:33:11.940 INFO:tasks.rados.rados.0.smithi178.stdout:335: done (5 left) 2024-04-26T04:33:11.940 INFO:tasks.rados.rados.0.smithi178.stdout:336: done (4 left) 2024-04-26T04:33:11.940 INFO:tasks.rados.rados.0.smithi178.stdout:340: done (3 left) 2024-04-26T04:33:11.940 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 11 current snap is 22 2024-04-26T04:33:11.940 INFO:tasks.rados.rados.0.smithi178.stdout:341: seq_num 137 ranges {180224=753664,933888=786432,1720320=655360,2375680=114688} 2024-04-26T04:33:11.950 INFO:tasks.rados.rados.0.smithi178.stdout:341: writing smithi17829308-11 from 180224 to 933888 tid 1 2024-04-26T04:33:11.958 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T04:33:11.960 INFO:tasks.rados.rados.0.smithi178.stdout:341: writing smithi17829308-11 from 933888 to 1720320 tid 2 2024-04-26T04:33:11.968 INFO:tasks.rados.rados.0.smithi178.stdout:341: writing smithi17829308-11 from 1720320 to 2375680 tid 3 2024-04-26T04:33:11.970 INFO:tasks.rados.rados.0.smithi178.stdout:341: writing smithi17829308-11 from 2375680 to 2490368 tid 4 2024-04-26T04:33:11.970 INFO:tasks.rados.rados.0.smithi178.stdout:342: copy_from oid 9 from oid 12 current snap is 22 2024-04-26T04:33:11.970 INFO:tasks.rados.rados.0.smithi178.stdout:343: setattr oid 45 current snap is 22 2024-04-26T04:33:11.973 INFO:tasks.rados.rados.0.smithi178.stdout:344: read oid 7 snap -1 2024-04-26T04:33:11.973 INFO:tasks.rados.rados.0.smithi178.stdout:344: expect deleted 2024-04-26T04:33:11.973 INFO:tasks.rados.rados.0.smithi178.stdout:345: snap_create 2024-04-26T04:33:11.989 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:11.990 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:12.003 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T04:33:12.008 INFO:tasks.rados.rados.0.smithi178.stdout:323: finishing rollback tid 2 to smithi17829308-18 2024-04-26T04:33:12.009 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 18 v 52 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-04-26T04:33:12.010 INFO:tasks.rados.rados.0.smithi178.stdout:326: finishing rollback tid 0 to smithi17829308-14 2024-04-26T04:33:12.033 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:33:12.033+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:33:12.123 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:33:12.123 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:33:12.133 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:12.133+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:12.135 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:12.135 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:12.143 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:33:12.144 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:33:12.150 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T04:33:12.163 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:33:12.163 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:33:12.184 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:33:12.184 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:33:12.201 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:33:12.201 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:33:12.234 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:12.230+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:12.264 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:33:12.264 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:33:12.272 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:12.272 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:12.272 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:33:12.272 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:33:12.272 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:12.277 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 45 v 84 (ObjNum 33 snap 0 seq_num 3640773057) dirty exists 2024-04-26T04:33:12.287 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T04:33:12.290 INFO:tasks.rados.rados.0.smithi178.stdout:323: done (7 left) 2024-04-26T04:33:12.290 INFO:tasks.rados.rados.0.smithi178.stdout:343: done (6 left) 2024-04-26T04:33:12.290 INFO:tasks.rados.rados.0.smithi178.stdout:344: done (5 left) 2024-04-26T04:33:12.290 INFO:tasks.rados.rados.0.smithi178.stdout:345: done (4 left) 2024-04-26T04:33:12.291 INFO:tasks.rados.rados.0.smithi178.stdout:346: read oid 30 snap -1 2024-04-26T04:33:12.291 INFO:tasks.rados.rados.0.smithi178.stdout:346: expect (ObjNum 103 snap 13 seq_num 103) 2024-04-26T04:33:12.291 INFO:tasks.rados.rados.0.smithi178.stdout:347: rollback oid 44 current snap is 23 2024-04-26T04:33:12.291 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 44 to 21 2024-04-26T04:33:12.291 INFO:tasks.rados.rados.0.smithi178.stdout:348: read oid 8 snap -1 2024-04-26T04:33:12.291 INFO:tasks.rados.rados.0.smithi178.stdout:348: expect (ObjNum 98 snap 12 seq_num 98) 2024-04-26T04:33:12.291 INFO:tasks.rados.rados.0.smithi178.stdout:349: read oid 49 snap 18 2024-04-26T04:33:12.291 INFO:tasks.rados.rados.0.smithi178.stdout:349: expect (ObjNum 117 snap 16 seq_num 117) 2024-04-26T04:33:12.291 INFO:tasks.rados.rados.0.smithi178.stdout:350: snap_remove snap 16 2024-04-26T04:33:12.295 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:33:12.295 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:33:12.325 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:33:12.325 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:33:12.335 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:33:12.333+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:33:12.411 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:33:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:33:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:33:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:42 1.0 e26/23 1:30306672:devicehealth::main.db.0000000000000000:head v 26'28, mlcod=26'26)", 2024-04-26T04:33:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:33:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 59.190139571000003, 2024-04-26T04:33:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201587136, 2024-04-26T04:33:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.412 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220676+0000", 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.843e-06 2024-04-26T04:33:12.413 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.414 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.414 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.414 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220687+0000", 2024-04-26T04:33:12.414 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1727e-05 2024-04-26T04:33:12.414 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.414 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.414 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.414 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220715+0000", 2024-04-26T04:33:12.414 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8033000000000001e-05 2024-04-26T04:33:12.414 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.414 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.414 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.414 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.320857+0000", 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014171400000001 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.421143+0000", 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100285902 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.421175+0000", 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2413999999999998e-05 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.415 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.422168+0000", 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0009931269999999999 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.422261+0000", 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2375999999999998e-05 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:55 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'33, mlcod=27'29)", 2024-04-26T04:33:12.416 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 58.866749708, 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.169473288, 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.417 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:33:12.418 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.418 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.418 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.418 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.418 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544065+0000", 2024-04-26T04:33:12.418 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.77e-06 2024-04-26T04:33:12.418 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.418 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.418 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.418 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544195+0000", 2024-04-26T04:33:12.418 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012971599999999999 2024-04-26T04:33:12.418 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544219+0000", 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3476e-05 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.644316+0000", 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100097189 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.711737+0000", 2024-04-26T04:33:12.419 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.067420846000000006 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.711770+0000", 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2959000000000001e-05 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.713428+0000", 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016584130000000001 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.420 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.713537+0000", 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010891900000000001 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:108 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'70, mlcod=27'64)", 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 55.602355545000002, 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30260933299999998, 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.421 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:33:12.422 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.37e-07 2024-04-26T04:33:12.423 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.423 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.423 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.423 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808464+0000", 2024-04-26T04:33:12.423 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8599999999999998e-06 2024-04-26T04:33:12.423 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.423 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.423 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808474+0000", 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.9830000000000008e-06 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908648+0000", 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10017390299999999 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908808+0000", 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000159576 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.424 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908845+0000", 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7026000000000001e-05 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110983+0000", 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20213835199999999 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111067+0000", 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3996e-05 2024-04-26T04:33:12.425 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.426 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.426 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.426 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.426 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.426 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:109 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'71, mlcod=27'64)", 2024-04-26T04:33:12.426 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:33:12.426 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 55.502120822000002, 2024-04-26T04:33:12.426 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202472819, 2024-04-26T04:33:12.426 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.426 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.426 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.426 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.426 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.427 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.427 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:33:12.427 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.427 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.427 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.427 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.427 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:33:12.427 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.427 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.427 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.427 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.427 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908694+0000", 2024-04-26T04:33:12.427 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.144e-06 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908751+0000", 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7268999999999997e-05 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908769+0000", 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8363e-05 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.428 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908780+0000", 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.096e-05 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.909341+0000", 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00056057500000000001 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.909371+0000", 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0357000000000001e-05 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.429 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111074+0000", 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20170292500000001 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111165+0000", 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.1225999999999997e-05 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:110 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'72, mlcod=27'64)", 2024-04-26T04:33:12.430 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:33:12.431 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 55.501985930000004, 2024-04-26T04:33:12.431 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20385624999999999, 2024-04-26T04:33:12.431 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.431 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.431 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.431 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.431 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.431 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.431 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:33:12.431 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.431 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.431 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.431 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908828+0000", 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9400000000000005e-07 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908836+0000", 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1929999999999993e-06 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.432 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908847+0000", 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0703000000000001e-05 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908855+0000", 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7829999999999999e-06 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009054+0000", 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10019966900000001 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.433 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.434 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.434 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009089+0000", 2024-04-26T04:33:12.434 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4433e-05 2024-04-26T04:33:12.434 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.434 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.434 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.434 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112582+0000", 2024-04-26T04:33:12.434 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103492973 2024-04-26T04:33:12.434 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.434 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.434 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.434 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112684+0000", 2024-04-26T04:33:12.434 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000101802 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:111 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'73, mlcod=27'64)", 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 55.501947000000001, 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.205422306, 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.435 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908867+0000", 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.03e-07 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.436 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908871+0000", 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5800000000000002e-06 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908883+0000", 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1507e-05 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009022+0000", 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100139593 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.437 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.438 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.438 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.109703+0000", 2024-04-26T04:33:12.438 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100680741 2024-04-26T04:33:12.438 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.438 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.438 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.438 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.109738+0000", 2024-04-26T04:33:12.438 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5296000000000002e-05 2024-04-26T04:33:12.438 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.438 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.438 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.438 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114207+0000", 2024-04-26T04:33:12.438 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0044682239999999998 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114289+0000", 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2062000000000001e-05 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:112 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'74, mlcod=27'64)", 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 55.401752878000003, 2024-04-26T04:33:12.439 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105317487, 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.440 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.441 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.441 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.441 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009061+0000", 2024-04-26T04:33:12.441 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.057e-06 2024-04-26T04:33:12.441 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.441 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.441 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.441 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009128+0000", 2024-04-26T04:33:12.441 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6859999999999993e-05 2024-04-26T04:33:12.441 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.441 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.441 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.441 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009147+0000", 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8196e-05 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009157+0000", 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0743e-05 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110254+0000", 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101096941 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.442 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.443 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110311+0000", 2024-04-26T04:33:12.443 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6718999999999997e-05 2024-04-26T04:33:12.443 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.443 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.443 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.443 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114296+0000", 2024-04-26T04:33:12.443 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0039847709999999998 2024-04-26T04:33:12.443 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.443 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.443 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.443 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114378+0000", 2024-04-26T04:33:12.443 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2200000000000006e-05 2024-04-26T04:33:12.443 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:114 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'76, mlcod=27'64)", 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 55.401603629, 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105329463, 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.444 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.53e-07 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.445 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009215+0000", 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7099999999999998e-06 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009223+0000", 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5599999999999994e-06 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009229+0000", 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0800000000000002e-06 2024-04-26T04:33:12.446 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111865+0000", 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102636071 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111941+0000", 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5615e-05 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114468+0000", 2024-04-26T04:33:12.447 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002526875 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114539+0000", 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1198999999999994e-05 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:115 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'77, mlcod=27'64)", 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 55.401575756, 2024-04-26T04:33:12.448 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105390922, 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.450 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.450 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.450 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:33:12.450 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.34e-07 2024-04-26T04:33:12.450 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.450 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.450 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.450 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009293+0000", 2024-04-26T04:33:12.450 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4954999999999998e-05 2024-04-26T04:33:12.450 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.450 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.450 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.450 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009304+0000", 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1263999999999999e-05 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009311+0000", 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6630000000000004e-06 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112802+0000", 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10349130600000001 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.451 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112843+0000", 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0472999999999997e-05 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114547+0000", 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001704552 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114628+0000", 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1374999999999999e-05 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.452 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:116 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'78, mlcod=27'64)", 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 55.401492503, 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105877241, 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.453 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.46e-07 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009326+0000", 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5040000000000004e-06 2024-04-26T04:33:12.454 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009334+0000", 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1929999999999993e-06 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009340+0000", 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.3729999999999998e-06 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.113595+0000", 2024-04-26T04:33:12.455 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104254889 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.113649+0000", 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3887000000000001e-05 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115105+0000", 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014560969999999999 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.456 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115198+0000", 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2851999999999998e-05 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:117 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'79, mlcod=27'64)", 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 55.401465020000003, 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10639752, 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.457 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.458 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.458 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.458 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:33:12.458 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.458 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.458 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.458 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:33:12.458 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.458 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.458 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.458 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009349+0000", 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7300000000000002e-07 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009353+0000", 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5970000000000002e-06 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009361+0000", 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0600000000000008e-06 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.459 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009381+0000", 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9533000000000001e-05 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114709+0000", 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105328753 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114745+0000", 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5976000000000002e-05 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.460 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115683+0000", 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00093774899999999998 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115746+0000", 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2578999999999998e-05 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:155 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'109, mlcod=27'105)", 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:33:12.461 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 53.571277246999998, 2024-04-26T04:33:12.462 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20562082700000001, 2024-04-26T04:33:12.462 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.462 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.462 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.462 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.462 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.462 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.462 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:33:12.462 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.462 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.462 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.462 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.462 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839537+0000", 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7400000000000003e-07 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839638+0000", 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000101365 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.463 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839653+0000", 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4874e-05 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939797+0000", 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014414100000001 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.040092+0000", 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100295305 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.464 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.040151+0000", 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8295999999999999e-05 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045057+0000", 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0049061390000000003 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045157+0000", 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000100133 2024-04-26T04:33:12.465 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:156 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'110, mlcod=27'105)", 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 53.470981770000002, 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10750937100000001, 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.466 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939833+0000", 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1030000000000001e-06 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.467 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939849+0000", 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6005999999999999e-05 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939866+0000", 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7513e-05 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939876+0000", 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0154e-05 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.468 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.041330+0000", 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101453864 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.041398+0000", 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7617e-05 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047245+0000", 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058470140000000002 2024-04-26T04:33:12.469 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.470 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.470 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.470 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047341+0000", 2024-04-26T04:33:12.470 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6100000000000005e-05 2024-04-26T04:33:12.470 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.470 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.470 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.470 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.470 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.470 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:157 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'111, mlcod=27'105)", 2024-04-26T04:33:12.470 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:33:12.470 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 53.470881630999997, 2024-04-26T04:33:12.470 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107487887, 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2300000000000001e-07 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939942+0000", 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4369999999999996e-06 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939952+0000", 2024-04-26T04:33:12.472 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.075e-05 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939963+0000", 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0069e-05 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.042203+0000", 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102240215 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.042259+0000", 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5955999999999998e-05 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047353+0000", 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0050939080000000003 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047420+0000", 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6928999999999996e-05 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:158 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'112, mlcod=27'105)", 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 53.470839044999998, 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.209867842, 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939975+0000", 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8599999999999999e-07 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940084+0000", 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000109746 2024-04-26T04:33:12.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940099+0000", 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4392999999999999e-05 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940110+0000", 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1236999999999999e-05 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043002+0000", 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102892262 2024-04-26T04:33:12.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043054+0000", 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1922999999999997e-05 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149774+0000", 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10672018599999999 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149842+0000", 2024-04-26T04:33:12.478 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7708999999999998e-05 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:159 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'113, mlcod=27'105)", 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 53.470690423000001, 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20976125000000001, 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.479 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7000000000000005e-07 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.480 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940131+0000", 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6369999999999999e-06 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940143+0000", 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1603000000000001e-05 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940165+0000", 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2042999999999999e-05 2024-04-26T04:33:12.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.482 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.482 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.482 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043777+0000", 2024-04-26T04:33:12.482 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103612145 2024-04-26T04:33:12.482 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.482 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.482 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.482 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043807+0000", 2024-04-26T04:33:12.482 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0562999999999999e-05 2024-04-26T04:33:12.482 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.482 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.482 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.482 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149845+0000", 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10603802900000001 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149884+0000", 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8659999999999999e-05 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:160 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'114, mlcod=27'105)", 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 53.470639917, 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20974973299999999, 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940174+0000", 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8999999999999998e-07 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940183+0000", 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6970000000000005e-06 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940194+0000", 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0983e-05 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940201+0000", 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4400000000000002e-06 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.044326+0000", 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104124984 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.044377+0000", 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1249999999999999e-05 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149886+0000", 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10550875999999999 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149923+0000", 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7329000000000001e-05 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:161 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'115, mlcod=27'105)", 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 53.470603973999999, 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.209755314, 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940210+0000", 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.67e-07 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940330+0000", 2024-04-26T04:33:12.490 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012061900000000001 2024-04-26T04:33:12.490 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.490 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.490 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.490 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940344+0000", 2024-04-26T04:33:12.490 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.367e-05 2024-04-26T04:33:12.490 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.490 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.490 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.490 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940355+0000", 2024-04-26T04:33:12.490 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.083e-05 2024-04-26T04:33:12.490 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.490 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045239+0000", 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104883873 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045278+0000", 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9904000000000003e-05 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149925+0000", 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104646221 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.491 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149965+0000", 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9929999999999999e-05 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:162 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'116, mlcod=27'105)", 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 53.467140241999999, 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.206336451, 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.492 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.493 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943675+0000", 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8640000000000001e-06 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943692+0000", 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7503000000000002e-05 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943720+0000", 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7569999999999999e-05 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.494 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943734+0000", 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3596e-05 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.046517+0000", 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102783897 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.046569+0000", 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1706000000000003e-05 2024-04-26T04:33:12.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149968+0000", 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10339902199999999 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150009+0000", 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1292999999999999e-05 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.496 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:163 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'117, mlcod=27'105)", 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 53.465639172000003, 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20488266699999999, 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.497 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945176+0000", 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8470000000000001e-06 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945193+0000", 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7013000000000001e-05 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.498 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945221+0000", 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7670000000000001e-05 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945238+0000", 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7113e-05 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.147614+0000", 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202375893 2024-04-26T04:33:12.499 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.147669+0000", 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5729000000000001e-05 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150011+0000", 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002342083 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150057+0000", 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5318999999999998e-05 2024-04-26T04:33:12.500 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.501 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.501 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.501 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.501 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.501 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:12.502 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:33:12.502 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:33:12.503 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:33:12.503 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:33:12.504 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T04:33:12.504 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:33:12.501+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:33:12.551 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:33:12.551 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:33:12.587 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:33:12.587 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:33:12.603 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:12.600+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:12.633 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:12.633 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:12.633 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:33:12.633 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:12.647 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T04:33:12.697 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:33:12.697 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:33:12.703 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:12.700+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:12.771 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:12.771 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:12.772 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:33:12.772 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:33:12.772 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:12.786 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T04:33:12.805 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:12.804+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:12.906 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:12.906 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:33:12.906 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:33:12.906 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:33:12.906 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.906 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 32/30 ECSubWrite(tid=0, reqid=client.4670.0:1, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:12.906 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.202677+0000", 2024-04-26T04:33:12.906 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 47.702126085000003, 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19503779500000001, 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202677+0000", 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202677+0000", 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.908 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.908 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.908 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202678+0000", 2024-04-26T04:33:12.908 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8599999999999998e-07 2024-04-26T04:33:12.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.908 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.908 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.908 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202736+0000", 2024-04-26T04:33:12.908 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7933000000000003e-05 2024-04-26T04:33:12.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.908 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.908 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.908 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202761+0000", 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4706999999999999e-05 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202786+0000", 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5528999999999999e-05 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.304898+0000", 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102111946 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.304915+0000", 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6846000000000001e-05 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397689+0000", 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.092774051999999996 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397715+0000", 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6196000000000001e-05 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.910 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.911 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.911 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.911 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.911 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 32/30 ECSubWrite(tid=1, reqid=client.4670.0:2, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:12.911 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.204544+0000", 2024-04-26T04:33:12.911 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 47.700259095, 2024-04-26T04:33:12.911 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19320447099999999, 2024-04-26T04:33:12.911 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.911 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.911 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.911 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.911 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.911 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204544+0000", 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204544+0000", 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204546+0000", 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.336e-06 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.912 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.913 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.913 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204575+0000", 2024-04-26T04:33:12.913 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9689999999999999e-05 2024-04-26T04:33:12.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.913 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.913 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.913 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.204597+0000", 2024-04-26T04:33:12.913 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1593000000000001e-05 2024-04-26T04:33:12.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.913 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.913 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.913 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.304678+0000", 2024-04-26T04:33:12.913 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10008115400000001 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305492+0000", 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000813666 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305505+0000", 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3169999999999999e-05 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.914 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397718+0000", 2024-04-26T04:33:12.915 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.092213081000000002 2024-04-26T04:33:12.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.915 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.915 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.915 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397749+0000", 2024-04-26T04:33:12.915 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0781000000000002e-05 2024-04-26T04:33:12.915 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.915 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.915 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.915 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.915 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.0s5 32/30 ECSubWrite(tid=0, reqid=client.4670.0:22, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:12.915 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.207578+0000", 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 47.697225805999999, 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19009067299999999, 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207578+0000", 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.916 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207578+0000", 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207579+0000", 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6270000000000001e-06 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207691+0000", 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000111382 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.917 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207728+0000", 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6946999999999997e-05 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207755+0000", 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7472999999999999e-05 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.297516+0000", 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.089760928000000004 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.918 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.297533+0000", 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7249999999999999e-05 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397639+0000", 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100105334 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397668+0000", 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9731999999999999e-05 2024-04-26T04:33:12.919 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.0s5 32/30 ECSubWrite(tid=1, reqid=client.4670.0:23, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.208255+0000", 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 47.696548272000001, 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.189431346, 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.920 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208255+0000", 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208255+0000", 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208257+0000", 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4869999999999999e-06 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.921 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208294+0000", 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7679999999999998e-05 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208325+0000", 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0533000000000001e-05 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208348+0000", 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2566999999999999e-05 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.922 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.298496+0000", 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.090148454000000003 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.298518+0000", 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2234000000000001e-05 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397672+0000", 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.099153924000000004 2024-04-26T04:33:12.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397687+0000", 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4467000000000001e-05 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 36/30 ECSubWrite(tid=39, reqid=client.4670.0:247, at_version=36'25, trim_to=0'0, roll_forward_to=35'24))", 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.074094+0000", 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 42.830709321999997, 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20298102000000001, 2024-04-26T04:33:12.924 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074094+0000", 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074094+0000", 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.925 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.926 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074096+0000", 2024-04-26T04:33:12.926 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.854e-06 2024-04-26T04:33:12.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.926 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.926 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.926 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074602+0000", 2024-04-26T04:33:12.926 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00050626899999999995 2024-04-26T04:33:12.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.926 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.926 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074642+0000", 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0113e-05 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174816+0000", 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10017388200000001 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174981+0000", 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016466500000000001 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174995+0000", 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.419e-05 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.277024+0000", 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102028498 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.277075+0000", 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1548999999999997e-05 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.928 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 40/30 ECSubWrite(tid=46, reqid=osd.5.0:46, at_version=40'31, trim_to=0'0, roll_forward_to=39'29))", 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.419586+0000", 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 38.485217110999997, 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10595267, 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.929 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419586+0000", 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419586+0000", 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419587+0000", 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5700000000000002e-07 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.930 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419629+0000", 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1628999999999997e-05 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419654+0000", 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5466999999999999e-05 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419670+0000", 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5699999999999999e-05 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.931 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419799+0000", 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000129085 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.419813+0000", 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4387e-05 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.525472+0000", 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105658988 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.932 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.525539+0000", 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6656999999999996e-05 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 40/30 ECSubWrite(tid=104, reqid=client.4670.0:321, at_version=40'18, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.611130+0000", 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 38.293673433000002, 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201625046, 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.933 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.934 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.934 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.934 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611130+0000", 2024-04-26T04:33:12.934 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.934 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.934 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.934 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.934 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611130+0000", 2024-04-26T04:33:12.934 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.934 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.934 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.934 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.934 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611131+0000", 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1300000000000001e-07 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611749+0000", 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00061894499999999998 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611779+0000", 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9969999999999999e-05 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.935 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611795+0000", 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5443000000000001e-05 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613563+0000", 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001768211 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613590+0000", 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6633000000000001e-05 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.936 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812660+0000", 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19907063799999999 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812755+0000", 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4792999999999999e-05 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 40/30 ECSubWrite(tid=106, reqid=client.4670.0:322, at_version=40'19, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:33:12.937 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.613168+0000", 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 38.291635118000002, 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.199657323, 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613168+0000", 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.938 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613168+0000", 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613174+0000", 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9569999999999999e-06 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613348+0000", 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000173535 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.939 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613377+0000", 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9522999999999998e-05 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613394+0000", 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.628e-05 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.615501+0000", 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021068710000000002 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.940 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.615545+0000", 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4406999999999997e-05 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812776+0000", 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19723128400000001 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812826+0000", 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9465999999999997e-05 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.941 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 40/30 ECSubWrite(tid=107, reqid=client.4670.0:323, at_version=40'20, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.613975+0000", 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 38.290828898000001, 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.199514147, 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.942 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613975+0000", 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613975+0000", 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613976+0000", 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8029999999999999e-06 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.943 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.614012+0000", 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5283000000000001e-05 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.614038+0000", 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5890000000000001e-05 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.614070+0000", 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1992999999999997e-05 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.944 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.945 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.945 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812143+0000", 2024-04-26T04:33:12.945 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.1980729 2024-04-26T04:33:12.945 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.945 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.945 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.945 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812157+0000", 2024-04-26T04:33:12.945 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.472e-05 2024-04-26T04:33:12.945 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.945 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.945 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.945 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.813409+0000", 2024-04-26T04:33:12.945 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012518290000000001 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.813489+0000", 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9728999999999995e-05 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.as5 47/30 ECSubWrite(tid=37, reqid=osd.3.0:37, at_version=47'29, trim_to=0'0, roll_forward_to=46'27))", 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:41.268922+0000", 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 31.635881718, 2024-04-26T04:33:12.946 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.140330859, 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.268922+0000", 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.268922+0000", 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.947 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.268922+0000", 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5299999999999999e-07 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.268962+0000", 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9589000000000003e-05 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.268983+0000", 2024-04-26T04:33:12.948 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0964000000000001e-05 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.269001+0000", 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8023e-05 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.407874+0000", 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.138872829 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.949 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.407902+0000", 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8663e-05 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.409171+0000", 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001268885 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:41.409253+0000", 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1452999999999994e-05 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.950 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.5s6 56/30 ECSubWrite(tid=63, reqid=client.4670.0:480, at_version=0'0, trim_to=0'0, roll_forward_to=56'44))", 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:50.926782+0000", 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.978021634000001, 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20180579100000001, 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926782+0000", 2024-04-26T04:33:12.951 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926782+0000", 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926784+0000", 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8530000000000001e-06 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.952 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926806+0000", 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2104e-05 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926835+0000", 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9635999999999999e-05 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.026977+0000", 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014184800000001 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.953 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.127264+0000", 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100287105 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.127291+0000", 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7046000000000002e-05 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.128508+0000", 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012162760000000001 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.954 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.955 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.955 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.128588+0000", 2024-04-26T04:33:12.955 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9923e-05 2024-04-26T04:33:12.955 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.955 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.955 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.955 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.955 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.957 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 60/30 ECSubWrite(tid=222, reqid=client.4670.0:597, at_version=60'46, trim_to=0'0, roll_forward_to=58'43))", 2024-04-26T04:33:12.957 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:55.982118+0000", 2024-04-26T04:33:12.957 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.922685260000002, 2024-04-26T04:33:12.957 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30298714199999999, 2024-04-26T04:33:12.957 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.957 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.957 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.957 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.957 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.957 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982118+0000", 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982118+0000", 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982119+0000", 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1969999999999999e-06 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.958 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982416+0000", 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000296087 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982433+0000", 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6986999999999998e-05 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982444+0000", 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1596e-05 2024-04-26T04:33:12.959 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.182651+0000", 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200207255 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.182661+0000", 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0017000000000001e-05 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.285076+0000", 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102415038 2024-04-26T04:33:12.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.285105+0000", 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8965000000000001e-05 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 60/30 ECSubWrite(tid=223, reqid=client.4670.0:598, at_version=60'47, trim_to=0'0, roll_forward_to=58'43))", 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:56.083100+0000", 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.821703719999999, 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202036665, 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.961 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083100+0000", 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083100+0000", 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.962 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083102+0000", 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2170000000000001e-06 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083219+0000", 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000117049 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083244+0000", 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4471e-05 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.963 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.183323+0000", 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100079945 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.183491+0000", 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016799800000000001 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.183505+0000", 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3740999999999999e-05 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.964 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.285116+0000", 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10161113500000001 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.285136+0000", 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0109e-05 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.965 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 61/30 ECSubWrite(tid=236, reqid=client.4670.0:599, at_version=0'0, trim_to=0'0, roll_forward_to=60'48))", 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:56.591417+0000", 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.313386807000001, 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200740057, 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.591417+0000", 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.966 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.591417+0000", 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.591418+0000", 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4890000000000001e-06 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.591439+0000", 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0653999999999999e-05 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.967 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.591463+0000", 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4223000000000002e-05 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.691580+0000", 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100116592 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.691750+0000", 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017070099999999999 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.968 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.691760+0000", 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8670000000000006e-06 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.792110+0000", 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100349255 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.792157+0000", 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7275999999999999e-05 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.969 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.5s6 67/30 ECSubWrite(tid=86, reqid=client.4670.0:732, at_version=67'59, trim_to=0'0, roll_forward_to=66'54))", 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:02.611645+0000", 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.293158432, 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20163228699999999, 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.970 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.611645+0000", 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.611645+0000", 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.611647+0000", 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7069999999999999e-06 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.971 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.611746+0000", 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.9269000000000001e-05 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.611773+0000", 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7076e-05 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.711865+0000", 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10009203899999999 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.972 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.812102+0000", 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100236353 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.812119+0000", 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7552999999999999e-05 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.813198+0000", 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001078437 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.973 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.813277+0000", 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9852999999999996e-05 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.es5 68/30 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=68'48, trim_to=0'0, roll_forward_to=66'46))", 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.242472+0000", 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 9.6623311699999999, 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201355757, 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.974 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.242472+0000", 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.242472+0000", 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.975 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.242473+0000", 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0699999999999997e-07 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.242526+0000", 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2706e-05 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.242550+0000", 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4036000000000002e-05 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.976 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.242570+0000", 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0653999999999999e-05 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243563+0000", 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00099253699999999993 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243577+0000", 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.447e-05 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.977 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.443743+0000", 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20016622100000001 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.443828+0000", 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.4625999999999999e-05 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.es5 68/30 ECSubWrite(tid=243, reqid=osd.13.0:243, at_version=68'50, trim_to=0'0, roll_forward_to=66'46))", 2024-04-26T04:33:12.978 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.243314+0000", 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 9.6614894810000003, 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20115462200000001, 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243314+0000", 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.979 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243314+0000", 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243316+0000", 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.773e-06 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243355+0000", 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8733999999999999e-05 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.980 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243385+0000", 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0372999999999999e-05 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243404+0000", 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8943000000000001e-05 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.442893+0000", 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.199489097 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.981 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.442924+0000", 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1142000000000001e-05 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.444396+0000", 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001472101 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.444469+0000", 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2459000000000005e-05 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.982 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 68/30 ECSubWrite(tid=273, reqid=osd.4.0:273, at_version=68'45, trim_to=0'0, roll_forward_to=66'43))", 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.438440+0000", 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 9.4663630869999995, 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105468433, 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.438440+0000", 2024-04-26T04:33:12.983 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.438440+0000", 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.438441+0000", 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5699999999999998e-07 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.438490+0000", 2024-04-26T04:33:12.984 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9527000000000002e-05 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.438520+0000", 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9416000000000001e-05 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.438542+0000", 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2643e-05 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.542158+0000", 2024-04-26T04:33:12.985 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103615892 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.542178+0000", 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9185999999999999e-05 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.543830+0000", 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001652316 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.986 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.543909+0000", 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8996000000000001e-05 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.es5 68/30 ECSubWrite(tid=244, reqid=osd.13.0:243, at_version=0'0, trim_to=0'0, roll_forward_to=68'50))", 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.645143+0000", 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 9.2596606490000006, 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20052077200000001, 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.987 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.645143+0000", 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.645143+0000", 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.645145+0000", 2024-04-26T04:33:12.988 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6869999999999999e-06 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.645166+0000", 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1046000000000001e-05 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.645195+0000", 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9104000000000001e-05 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.645210+0000", 2024-04-26T04:33:12.989 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5662999999999999e-05 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.844505+0000", 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19929448499999999 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.844520+0000", 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4786999999999999e-05 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.990 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.845589+0000", 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001069637 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.845664+0000", 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4363000000000006e-05 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.5s6 68/30 ECSubWrite(tid=91, reqid=osd.15.0:90, at_version=0'0, trim_to=0'0, roll_forward_to=68'63))", 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.835304+0000", 2024-04-26T04:33:12.991 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 9.0694996870000004, 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20152138999999999, 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.835304+0000", 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.835304+0000", 2024-04-26T04:33:12.992 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:12.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.993 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.993 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:12.993 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.835306+0000", 2024-04-26T04:33:12.993 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7999999999999999e-06 2024-04-26T04:33:12.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.993 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.993 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:12.993 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.835327+0000", 2024-04-26T04:33:12.993 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1250000000000002e-05 2024-04-26T04:33:12.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.993 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.835358+0000", 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.065e-05 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.935467+0000", 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100109771 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.035772+0000", 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100304755 2024-04-26T04:33:12.994 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.995 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.995 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:12.995 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.035787+0000", 2024-04-26T04:33:12.995 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4953000000000001e-05 2024-04-26T04:33:12.995 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.995 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.995 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:12.995 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.036756+0000", 2024-04-26T04:33:12.995 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00096854099999999995 2024-04-26T04:33:12.995 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:12.996 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:12.996 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:12.996 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.036825+0000", 2024-04-26T04:33:12.996 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9670000000000002e-05 2024-04-26T04:33:12.996 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.996 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.996 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.996 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:12.996 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:12.996 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:12.997 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:12.905+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:12.997 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:33:12.998 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:33:12.998 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:33:12.999 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T04:33:12.999 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:33:12.999 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:33:13.000 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:33:13.007 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:13.004+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:13.109 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:13.105+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:13.126 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:13.126 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:13.126 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:33:13.126 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:13.140 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T04:33:13.201 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-26T04:33:13.245 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:13.245+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:13.263 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:13.263 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:13.263 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:33:13.263 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:33:13.263 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:13.277 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T04:33:13.279 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [2] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T04:33:13.279 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T04:33:13.279 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-04-26T04:33:13.280 INFO:tasks.ceph.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 16 2024-04-26T04:33:13.280 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:33:13.296 INFO:tasks.rados.rados.0.smithi178.stdout:326: finishing rollback tid 1 to smithi17829308-14 2024-04-26T04:33:13.297 INFO:tasks.rados.rados.0.smithi178.stdout:326: finishing rollback tid 2 to smithi17829308-14 2024-04-26T04:33:13.297 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 14 v 55 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-04-26T04:33:13.297 INFO:tasks.rados.rados.0.smithi178.stdout:339: finishing write tid 1 to smithi17829308-19 2024-04-26T04:33:13.297 INFO:tasks.rados.rados.0.smithi178.stdout:342: finishing copy_from racing read to smithi17829308-9 2024-04-26T04:33:13.297 INFO:tasks.rados.rados.0.smithi178.stdout:339: finishing write tid 2 to smithi17829308-19 2024-04-26T04:33:13.297 INFO:tasks.rados.rados.0.smithi178.stdout:339: finishing write tid 3 to smithi17829308-19 2024-04-26T04:33:13.297 INFO:tasks.rados.rados.0.smithi178.stdout:339: finishing write tid 4 to smithi17829308-19 2024-04-26T04:33:13.297 INFO:tasks.rados.rados.0.smithi178.stdout:339: finishing write tid 5 to smithi17829308-19 2024-04-26T04:33:13.297 INFO:tasks.rados.rados.0.smithi178.stdout:342: finishing copy_from to smithi17829308-9 2024-04-26T04:33:13.297 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 9 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-04-26T04:33:13.297 INFO:tasks.rados.rados.0.smithi178.stdout:339: finishing write tid 6 to smithi17829308-19 2024-04-26T04:33:13.297 INFO:tasks.rados.rados.0.smithi178.stdout:339: oid 19 updating version 0 to 64 2024-04-26T04:33:13.297 INFO:tasks.rados.rados.0.smithi178.stdout:339: oid 19 version 64 is already newer than 62 2024-04-26T04:33:13.298 INFO:tasks.rados.rados.0.smithi178.stdout:339: oid 19 version 64 is already newer than 60 2024-04-26T04:33:13.298 INFO:tasks.rados.rados.0.smithi178.stdout:339: oid 19 version 64 is already newer than 61 2024-04-26T04:33:13.298 INFO:tasks.rados.rados.0.smithi178.stdout:339: oid 19 version 64 is already newer than 63 2024-04-26T04:33:13.298 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 19 v 64 (ObjNum 135 snap 22 seq_num 135) dirty exists 2024-04-26T04:33:13.298 INFO:tasks.rados.rados.0.smithi178.stdout:339: left oid 19 (ObjNum 135 snap 22 seq_num 135) 2024-04-26T04:33:13.298 INFO:tasks.rados.rados.0.smithi178.stdout:347: finishing rollback tid 0 to smithi17829308-44 2024-04-26T04:33:13.298 INFO:tasks.rados.rados.0.smithi178.stdout:348: expect (ObjNum 98 snap 12 seq_num 98) 2024-04-26T04:33:13.310 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:13.306+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:13.392 INFO:tasks.rados.rados.0.smithi178.stdout:347: finishing rollback tid 1 to smithi17829308-44 2024-04-26T04:33:13.392 INFO:tasks.rados.rados.0.smithi178.stdout:347: finishing rollback tid 2 to smithi17829308-44 2024-04-26T04:33:13.392 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 44 v 86 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-26T04:33:13.392 INFO:tasks.rados.rados.0.smithi178.stdout:346: expect (ObjNum 103 snap 13 seq_num 103) 2024-04-26T04:33:13.404 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:13.404 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 35/30 ECSubWrite(tid=61, reqid=client.4670.0:211, at_version=0'0, trim_to=0'0, roll_forward_to=35'17))", 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:29.330344+0000", 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 44.074788753, 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105013624, 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330344+0000", 2024-04-26T04:33:13.405 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330344+0000", 2024-04-26T04:33:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330346+0000", 2024-04-26T04:33:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1399999999999998e-06 2024-04-26T04:33:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.406 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330371+0000", 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5320000000000002e-05 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330403+0000", 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2305999999999999e-05 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330423+0000", 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9179000000000001e-05 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.407 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.433931+0000", 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103508818 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.433962+0000", 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0576e-05 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.435260+0000", 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001297819 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.408 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.435357+0000", 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.7466000000000005e-05 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=73, reqid=client.4670.0:259, at_version=36'27, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.257763+0000", 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 43.147369068000003, 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103026436, 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.409 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.257763+0000", 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.257763+0000", 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.257766+0000", 2024-04-26T04:33:13.410 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.216e-06 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258080+0000", 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00031390699999999999 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258110+0000", 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0497e-05 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258127+0000", 2024-04-26T04:33:13.411 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6663e-05 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.358401+0000", 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10027383099999999 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.358415+0000", 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4207e-05 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.360708+0000", 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0022931499999999999 2024-04-26T04:33:13.412 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.360790+0000", 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1965000000000006e-05 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=74, reqid=client.4670.0:260, at_version=36'28, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.258165+0000", 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 43.146967642, 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10329530100000001, 2024-04-26T04:33:13.413 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258165+0000", 2024-04-26T04:33:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.414 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258165+0000", 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258166+0000", 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8299999999999995e-07 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258269+0000", 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000103069 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.415 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258288+0000", 2024-04-26T04:33:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8652999999999999e-05 2024-04-26T04:33:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258299+0000", 2024-04-26T04:33:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1243999999999999e-05 2024-04-26T04:33:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.359776+0000", 2024-04-26T04:33:13.416 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10147722000000001 2024-04-26T04:33:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.359798+0000", 2024-04-26T04:33:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2476999999999998e-05 2024-04-26T04:33:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.361395+0000", 2024-04-26T04:33:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0015963660000000001 2024-04-26T04:33:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.417 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.361460+0000", 2024-04-26T04:33:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5289000000000005e-05 2024-04-26T04:33:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 41/30 ECSubWrite(tid=44, reqid=client.4670.0:346, at_version=41'26, trim_to=0'0, roll_forward_to=38'24))", 2024-04-26T04:33:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:33:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 37.616844585000003, 2024-04-26T04:33:13.418 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103457721, 2024-04-26T04:33:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:33:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.419 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:33:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:33:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4600000000000005e-07 2024-04-26T04:33:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788918+0000", 2024-04-26T04:33:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00062948799999999999 2024-04-26T04:33:13.420 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788943+0000", 2024-04-26T04:33:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4706e-05 2024-04-26T04:33:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.889107+0000", 2024-04-26T04:33:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100164644 2024-04-26T04:33:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.421 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.889285+0000", 2024-04-26T04:33:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017809600000000001 2024-04-26T04:33:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.889311+0000", 2024-04-26T04:33:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5233e-05 2024-04-26T04:33:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.422 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.891664+0000", 2024-04-26T04:33:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0023538460000000001 2024-04-26T04:33:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.891746+0000", 2024-04-26T04:33:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1162000000000007e-05 2024-04-26T04:33:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.423 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/30 ECSubWrite(tid=186, reqid=client.4670.0:483, at_version=57'57, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.517139+0000", 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.887993932000001, 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.22991919599999999, 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.517139+0000", 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.424 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.517139+0000", 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.517139+0000", 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6700000000000003e-07 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.518064+0000", 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00092505799999999998 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.425 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.518095+0000", 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0929999999999997e-05 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.518120+0000", 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4888999999999998e-05 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.644583+0000", 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.12646262999999999 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.426 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.427 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.427 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.644598+0000", 2024-04-26T04:33:13.427 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5427e-05 2024-04-26T04:33:13.427 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.427 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.427 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.427 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747001+0000", 2024-04-26T04:33:13.427 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102403228 2024-04-26T04:33:13.427 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.427 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.427 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.427 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747058+0000", 2024-04-26T04:33:13.427 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6366999999999998e-05 2024-04-26T04:33:13.428 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.428 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.428 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.428 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.428 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.428 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/30 ECSubWrite(tid=187, reqid=client.4670.0:484, at_version=57'58, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T04:33:13.428 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.519747+0000", 2024-04-26T04:33:13.428 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.885385535000001, 2024-04-26T04:33:13.428 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.22735620400000001, 2024-04-26T04:33:13.428 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.428 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.428 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.428 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.519747+0000", 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.519747+0000", 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.519764+0000", 2024-04-26T04:33:13.429 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7249999999999999e-05 2024-04-26T04:33:13.430 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.430 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.430 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.430 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.520091+0000", 2024-04-26T04:33:13.430 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000326457 2024-04-26T04:33:13.430 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.430 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.430 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.430 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.520114+0000", 2024-04-26T04:33:13.430 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3462999999999999e-05 2024-04-26T04:33:13.430 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.430 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.431 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.431 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.520135+0000", 2024-04-26T04:33:13.431 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0449999999999999e-05 2024-04-26T04:33:13.431 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.431 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.431 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.431 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.646545+0000", 2024-04-26T04:33:13.431 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.12641021699999999 2024-04-26T04:33:13.431 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.431 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.431 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.431 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.646559+0000", 2024-04-26T04:33:13.431 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4137e-05 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747071+0000", 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100512381 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747103+0000", 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1848999999999999e-05 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.432 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.433 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.433 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/30 ECSubWrite(tid=188, reqid=client.4670.0:485, at_version=57'59, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T04:33:13.433 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.521190+0000", 2024-04-26T04:33:13.433 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.883942434000001, 2024-04-26T04:33:13.433 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.225943687, 2024-04-26T04:33:13.433 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.433 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.433 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.433 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.433 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.433 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.433 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521190+0000", 2024-04-26T04:33:13.433 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.434 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.434 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.434 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.434 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521190+0000", 2024-04-26T04:33:13.434 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.434 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.434 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.434 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.434 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521191+0000", 2024-04-26T04:33:13.434 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2500000000000001e-06 2024-04-26T04:33:13.434 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.434 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.434 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521409+0000", 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000218192 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521431+0000", 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1776e-05 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621545+0000", 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10011374300000001 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.435 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.647483+0000", 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.025938086999999999 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.647497+0000", 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.376e-05 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747108+0000", 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.099611379 2024-04-26T04:33:13.436 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747134+0000", 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.55e-05 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/30 ECSubWrite(tid=189, reqid=client.4670.0:486, at_version=57'60, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.621601+0000", 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.783531722999999, 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.126106461, 2024-04-26T04:33:13.437 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621601+0000", 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621601+0000", 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.438 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621602+0000", 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.576e-06 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621645+0000", 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2620000000000002e-05 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621689+0000", 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4502999999999997e-05 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.439 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621712+0000", 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2486000000000001e-05 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.746683+0000", 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.124971128 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.746698+0000", 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5140000000000001e-05 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.440 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747655+0000", 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00095687099999999998 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747707+0000", 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2136999999999999e-05 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.441 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 58/30 ECSubWrite(tid=195, reqid=osd.4.0:195, at_version=58'62, trim_to=0'0, roll_forward_to=57'60))", 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:52.382786+0000", 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 21.022346274, 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20206534200000001, 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382786+0000", 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.442 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382786+0000", 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382787+0000", 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9000000000000002e-07 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382822+0000", 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5889999999999997e-05 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.443 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382851+0000", 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8370000000000001e-05 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.482997+0000", 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100146258 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.583261+0000", 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100264055 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.444 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.445 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.583277+0000", 2024-04-26T04:33:13.445 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5783000000000001e-05 2024-04-26T04:33:13.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.445 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.445 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.445 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.584773+0000", 2024-04-26T04:33:13.445 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014958440000000001 2024-04-26T04:33:13.445 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.445 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.445 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.445 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.584852+0000", 2024-04-26T04:33:13.445 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8751999999999994e-05 2024-04-26T04:33:13.445 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpReadReply(3.2s1 61/60 ECSubReadReply(tid=2, attrs_read=0))", 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:56.636057+0000", 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.769075870999998, 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200517522, 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.446 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636057+0000", 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636057+0000", 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636058+0000", 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1429999999999999e-06 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.447 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636180+0000", 2024-04-26T04:33:13.448 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000122409 2024-04-26T04:33:13.448 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.448 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.448 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.448 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636199+0000", 2024-04-26T04:33:13.448 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8536999999999999e-05 2024-04-26T04:33:13.448 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.448 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.448 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.448 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.736272+0000", 2024-04-26T04:33:13.448 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100073096 2024-04-26T04:33:13.448 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.448 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.836499+0000", 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10022764100000001 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.836574+0000", 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4696000000000004e-05 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.449 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 67/30 ECSubWrite(tid=65, reqid=client.4670.0:742, at_version=67'43, trim_to=0'0, roll_forward_to=65'40))", 2024-04-26T04:33:13.450 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:02.478292+0000", 2024-04-26T04:33:13.450 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.926840692000001, 2024-04-26T04:33:13.450 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102970543, 2024-04-26T04:33:13.450 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.450 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.450 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.450 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.450 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.450 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.450 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.478292+0000", 2024-04-26T04:33:13.450 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.451 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.451 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.451 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.478292+0000", 2024-04-26T04:33:13.451 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.451 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.451 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.451 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.478294+0000", 2024-04-26T04:33:13.451 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.866e-06 2024-04-26T04:33:13.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.451 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.451 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.478500+0000", 2024-04-26T04:33:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000206288 2024-04-26T04:33:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.478525+0000", 2024-04-26T04:33:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4576999999999999e-05 2024-04-26T04:33:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.578695+0000", 2024-04-26T04:33:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100170441 2024-04-26T04:33:13.452 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.453 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.453 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.453 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.578861+0000", 2024-04-26T04:33:13.453 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016581199999999999 2024-04-26T04:33:13.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.453 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.453 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.453 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.578889+0000", 2024-04-26T04:33:13.453 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.792e-05 2024-04-26T04:33:13.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.453 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.453 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.454 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.581194+0000", 2024-04-26T04:33:13.454 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002305156 2024-04-26T04:33:13.454 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.454 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.454 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.454 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.581262+0000", 2024-04-26T04:33:13.454 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.8483000000000001e-05 2024-04-26T04:33:13.454 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.454 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.454 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.454 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.454 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.454 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 67/30 ECSubWrite(tid=66, reqid=client.4670.0:743, at_version=67'44, trim_to=0'0, roll_forward_to=65'40))", 2024-04-26T04:33:13.455 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:02.578744+0000", 2024-04-26T04:33:13.455 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.8263889, 2024-04-26T04:33:13.455 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10783959799999999, 2024-04-26T04:33:13.455 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.455 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.455 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.455 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.455 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.455 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.455 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.578744+0000", 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.578744+0000", 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.578745+0000", 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.113e-06 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.456 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.457 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.578975+0000", 2024-04-26T04:33:13.457 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00023011099999999999 2024-04-26T04:33:13.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.457 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.457 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.457 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.578996+0000", 2024-04-26T04:33:13.457 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0996999999999999e-05 2024-04-26T04:33:13.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.457 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.457 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.579015+0000", 2024-04-26T04:33:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8966000000000001e-05 2024-04-26T04:33:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583750+0000", 2024-04-26T04:33:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0047352059999999996 2024-04-26T04:33:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583767+0000", 2024-04-26T04:33:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7436999999999999e-05 2024-04-26T04:33:13.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.459 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.459 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.459 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.686514+0000", 2024-04-26T04:33:13.459 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10274667899999999 2024-04-26T04:33:13.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.459 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.459 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.459 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.686583+0000", 2024-04-26T04:33:13.459 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9089000000000002e-05 2024-04-26T04:33:13.459 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.459 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.460 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.460 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.460 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.460 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 67/30 ECSubWrite(tid=67, reqid=client.4670.0:744, at_version=67'45, trim_to=0'0, roll_forward_to=65'40))", 2024-04-26T04:33:13.460 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:02.583506+0000", 2024-04-26T04:33:13.460 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.821626975000001, 2024-04-26T04:33:13.460 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10420913699999999, 2024-04-26T04:33:13.460 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.460 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.460 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.460 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.460 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583506+0000", 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583506+0000", 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583507+0000", 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8199999999999999e-06 2024-04-26T04:33:13.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583576+0000", 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9103000000000006e-05 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583606+0000", 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9626000000000001e-05 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583622+0000", 2024-04-26T04:33:13.462 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5756999999999998e-05 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.685678+0000", 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102056015 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.685693+0000", 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4943000000000001e-05 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.463 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.687636+0000", 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001943033 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.687715+0000", 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8839999999999997e-05 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 67/30 ECSubWrite(tid=68, reqid=client.4670.0:745, at_version=67'46, trim_to=0'0, roll_forward_to=65'40))", 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:02.585301+0000", 2024-04-26T04:33:13.464 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.819831087000001, 2024-04-26T04:33:13.465 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10331043099999999, 2024-04-26T04:33:13.465 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.465 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.465 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.465 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.465 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.465 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.465 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.585301+0000", 2024-04-26T04:33:13.465 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.465 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.465 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.465 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.465 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.585301+0000", 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.585303+0000", 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.79e-06 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.585338+0000", 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4459000000000003e-05 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.466 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.585363+0000", 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5063999999999999e-05 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.585380+0000", 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7212999999999999e-05 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.686687+0000", 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10130702799999999 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.467 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.468 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.468 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.686714+0000", 2024-04-26T04:33:13.468 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7379999999999999e-05 2024-04-26T04:33:13.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.468 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.468 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.468 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.688535+0000", 2024-04-26T04:33:13.468 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001820781 2024-04-26T04:33:13.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.468 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.468 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.468 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.688612+0000", 2024-04-26T04:33:13.468 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6716000000000002e-05 2024-04-26T04:33:13.469 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.469 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.469 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.469 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.469 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.469 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 68/30 ECSubWrite(tid=271, reqid=osd.4.0:271, at_version=68'78, trim_to=0'0, roll_forward_to=65'76))", 2024-04-26T04:33:13.469 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.337687+0000", 2024-04-26T04:33:13.469 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.067445033, 2024-04-26T04:33:13.469 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103882078, 2024-04-26T04:33:13.469 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.469 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.337687+0000", 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.337687+0000", 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.470 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.337688+0000", 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9999999999999998e-07 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.337711+0000", 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3456e-05 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.337729+0000", 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7631000000000001e-05 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.471 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.337739+0000", 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8239999999999995e-06 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.440088+0000", 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102349147 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.440106+0000", 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.855e-05 2024-04-26T04:33:13.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.441496+0000", 2024-04-26T04:33:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001389728 2024-04-26T04:33:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.441570+0000", 2024-04-26T04:33:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3342000000000004e-05 2024-04-26T04:33:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.474 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.474 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 68/30 ECSubWrite(tid=71, reqid=osd.7.0:71, at_version=68'48, trim_to=0'0, roll_forward_to=67'46))", 2024-04-26T04:33:13.474 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.702942+0000", 2024-04-26T04:33:13.474 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 9.7021903799999993, 2024-04-26T04:33:13.474 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104726319, 2024-04-26T04:33:13.474 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.474 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.474 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.474 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.474 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.474 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.474 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.702942+0000", 2024-04-26T04:33:13.474 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.475 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.475 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.475 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.475 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.702942+0000", 2024-04-26T04:33:13.475 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.475 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.475 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.475 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.475 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.702944+0000", 2024-04-26T04:33:13.475 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.59e-06 2024-04-26T04:33:13.475 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.475 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.475 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.702991+0000", 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7519999999999999e-05 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.703025+0000", 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3577000000000001e-05 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.703047+0000", 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2682000000000001e-05 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.476 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.803360+0000", 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100312097 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.803375+0000", 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4949999999999999e-05 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.807578+0000", 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0042032199999999997 2024-04-26T04:33:13.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.807668+0000", 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.0682999999999999e-05 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 68/30 ECSubWrite(tid=72, reqid=osd.7.0:72, at_version=68'50, trim_to=0'0, roll_forward_to=67'46))", 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.705213+0000", 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 9.6999196790000006, 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10635373400000001, 2024-04-26T04:33:13.478 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705213+0000", 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705213+0000", 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.479 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705215+0000", 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9630000000000001e-06 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705265+0000", 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0073e-05 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705303+0000", 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8206e-05 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.480 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705326+0000", 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3453000000000001e-05 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.805555+0000", 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100228715 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.805585+0000", 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9828999999999999e-05 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.481 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.482 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.482 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.811484+0000", 2024-04-26T04:33:13.482 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058986029999999997 2024-04-26T04:33:13.482 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.482 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.482 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.482 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.811567+0000", 2024-04-26T04:33:13.482 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2892000000000005e-05 2024-04-26T04:33:13.482 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.482 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.482 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.482 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.482 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 68/30 ECSubWrite(tid=73, reqid=osd.7.0:72, at_version=0'0, trim_to=0'0, roll_forward_to=68'50))", 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:04.010174+0000", 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 9.3949582879999998, 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201687914, 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.010174+0000", 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.483 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.010174+0000", 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.010177+0000", 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0429999999999999e-06 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.010199+0000", 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1494000000000001e-05 2024-04-26T04:33:13.484 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.485 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.485 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.485 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.010229+0000", 2024-04-26T04:33:13.485 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0759999999999997e-05 2024-04-26T04:33:13.485 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.485 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.485 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.485 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.110375+0000", 2024-04-26T04:33:13.485 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014508599999999 2024-04-26T04:33:13.485 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.485 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.485 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.485 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.210624+0000", 2024-04-26T04:33:13.486 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100249067 2024-04-26T04:33:13.486 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.486 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.486 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.486 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.210641+0000", 2024-04-26T04:33:13.486 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7295999999999998e-05 2024-04-26T04:33:13.486 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.486 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.486 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.486 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.211820+0000", 2024-04-26T04:33:13.486 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001179474 2024-04-26T04:33:13.486 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.486 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.211862+0000", 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1693999999999998e-05 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 72/30 ECSubWrite(tid=120, reqid=client.4670.0:818, at_version=72'83, trim_to=0'0, roll_forward_to=71'80))", 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:07.490773+0000", 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.9143597879999996, 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105442548, 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.487 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.490773+0000", 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.490773+0000", 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.488 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.490774+0000", 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7400000000000001e-06 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.491052+0000", 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00027784799999999997 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.491082+0000", 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9573e-05 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.489 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.490 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.491098+0000", 2024-04-26T04:33:13.490 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6393e-05 2024-04-26T04:33:13.490 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.490 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.490 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.490 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.588489+0000", 2024-04-26T04:33:13.490 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.097390657000000005 2024-04-26T04:33:13.490 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.490 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.588504+0000", 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5107e-05 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.596157+0000", 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0076531239999999999 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.596215+0000", 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8106000000000002e-05 2024-04-26T04:33:13.491 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.492 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.492 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.492 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.492 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.492 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 72/30 ECSubWrite(tid=123, reqid=client.4670.0:821, at_version=72'86, trim_to=0'0, roll_forward_to=71'80))", 2024-04-26T04:33:13.492 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:07.689310+0000", 2024-04-26T04:33:13.492 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.715822406, 2024-04-26T04:33:13.492 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201647774, 2024-04-26T04:33:13.492 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.492 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.492 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:13.492 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.492 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.493 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.493 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.689310+0000", 2024-04-26T04:33:13.493 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.493 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.493 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.493 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.493 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.689310+0000", 2024-04-26T04:33:13.493 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.493 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.493 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.493 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.494 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.689311+0000", 2024-04-26T04:33:13.494 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6000000000000003e-07 2024-04-26T04:33:13.494 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.494 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.494 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.494 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.689353+0000", 2024-04-26T04:33:13.494 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2518999999999997e-05 2024-04-26T04:33:13.494 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.494 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.494 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.494 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.689367+0000", 2024-04-26T04:33:13.495 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3720000000000001e-05 2024-04-26T04:33:13.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.495 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.495 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.495 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.789502+0000", 2024-04-26T04:33:13.495 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10013504300000001 2024-04-26T04:33:13.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.495 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.495 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.495 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.889740+0000", 2024-04-26T04:33:13.495 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10023826199999999 2024-04-26T04:33:13.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.496 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.496 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.496 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.889755+0000", 2024-04-26T04:33:13.496 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4763000000000001e-05 2024-04-26T04:33:13.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.496 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.496 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.496 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.890881+0000", 2024-04-26T04:33:13.496 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0011258570000000001 2024-04-26T04:33:13.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.496 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.497 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.497 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.890958+0000", 2024-04-26T04:33:13.497 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6849999999999998e-05 2024-04-26T04:33:13.497 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.497 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.497 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.497 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.497 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.497 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:13.498 INFO:tasks.rados.rados.0.smithi178.stdout:341: finishing write tid 1 to smithi17829308-11 2024-04-26T04:33:13.499 INFO:tasks.rados.rados.0.smithi178.stdout:341: finishing write tid 2 to smithi17829308-11 2024-04-26T04:33:13.499 INFO:tasks.rados.rados.0.smithi178.stdout:349: expect (ObjNum 117 snap 16 seq_num 117) 2024-04-26T04:33:13.499 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:13.494+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:13.500 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T04:33:13.515 INFO:tasks.rados.rados.0.smithi178.stdout:341: finishing write tid 3 to smithi17829308-11 2024-04-26T04:33:13.515 INFO:tasks.rados.rados.0.smithi178.stdout:341: finishing write tid 4 to smithi17829308-11 2024-04-26T04:33:13.515 INFO:tasks.rados.rados.0.smithi178.stdout:341: finishing write tid 5 to smithi17829308-11 2024-04-26T04:33:13.516 INFO:tasks.rados.rados.0.smithi178.stdout:326: done (8 left) 2024-04-26T04:33:13.516 INFO:tasks.rados.rados.0.smithi178.stdout:339: done (7 left) 2024-04-26T04:33:13.516 INFO:tasks.rados.rados.0.smithi178.stdout:342: done (6 left) 2024-04-26T04:33:13.516 INFO:tasks.rados.rados.0.smithi178.stdout:346: done (5 left) 2024-04-26T04:33:13.516 INFO:tasks.rados.rados.0.smithi178.stdout:347: done (4 left) 2024-04-26T04:33:13.516 INFO:tasks.rados.rados.0.smithi178.stdout:348: done (3 left) 2024-04-26T04:33:13.516 INFO:tasks.rados.rados.0.smithi178.stdout:349: done (2 left) 2024-04-26T04:33:13.516 INFO:tasks.rados.rados.0.smithi178.stdout:350: done (1 left) 2024-04-26T04:33:13.516 INFO:tasks.rados.rados.0.smithi178.stdout:351: read oid 27 snap -1 2024-04-26T04:33:13.516 INFO:tasks.rados.rados.0.smithi178.stdout:351: expect (ObjNum 87 snap 12 seq_num 87) 2024-04-26T04:33:13.516 INFO:tasks.rados.rados.0.smithi178.stdout:352: snap_remove snap 17 2024-04-26T04:33:13.595 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:33:13.596 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:33:13.599 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:13.596+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:13.613 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-04-26T04:33:13.343916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":93422112,"num_objects":77,"num_object_clones":29,"num_object_copies":529,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":3,"num_read":600,"num_read_kb":223190,"num_write":875,"num_write_kb":120276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":983,"ondisk_log_size":983,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":15,"num_per_pool_osds":15,"num_per_pool_omap_osds":15,"kb":1415577600,"kb_used":630240,"kb_used_data":174988,"kb_used_omap":23,"kb_used_meta":455016,"kb_avail":1414947360,"statfs":{"total":1449551462400,"available":1448906096640,"internally_reserved":0,"allocated":179187712,"data_stored":213598952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23926,"internal_metadata":465937034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,16,4,5,10],"upper_bound":512},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9535488,"num_objects":9,"num_object_clones":8,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":-1,"num_read":33,"num_read_kb":38126,"num_write":69,"num_write_kb":9312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001359"},"pg_stats":[{"pgid":"3.f","version":"74'91","reported_seq":305,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-26T04:33:09.275073+0000","last_change":"2024-04-26T04:33:04.850194+0000","last_active":"2024-04-26T04:33:09.275073+0000","last_peered":"2024-04-26T04:33:09.275073+0000","last_clean":"2024-04-26T04:33:09.275073+0000","last_became_active":"2024-04-26T04:32:23.378967+0000","last_became_peered":"2024-04-26T04:32:23.378967+0000","last_unstale":"2024-04-26T04:33:09.275073+0000","last_undegraded":"2024-04-26T04:33:09.275073+0000","last_fullsized":"2024-04-26T04:33:09.275073+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:45.684473+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.50966482499999999,"stat_sum":{"num_bytes":8552448,"num_objects":10,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":53,"num_read_kb":7340,"num_write":99,"num_write_kb":12976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,14,13,8,10,0],"acting":[11,1,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"3.e","version":"73'57","reported_seq":257,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-26T04:33:09.375949+0000","last_change":"2024-04-26T04:33:04.952481+0000","last_active":"2024-04-26T04:33:09.375949+0000","last_peered":"2024-04-26T04:33:09.375949+0000","last_clean":"2024-04-26T04:33:09.375949+0000","last_became_active":"2024-04-26T04:32:23.387938+0000","last_became_peered":"2024-04-26T04:32:23.387938+0000","last_unstale":"2024-04-26T04:33:09.375949+0000","last_undegraded":"2024-04-26T04:33:09.375949+0000","last_fullsized":"2024-04-26T04:33:09.375949+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:32.283826+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.38581605600000002,"stat_sum":{"num_bytes":7913472,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":22,"num_read_kb":24202,"num_write":31,"num_write_kb":8032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,9,1,11,8,6,12],"acting":[13,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"3.d","version":"70'58","reported_seq":266,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-26T04:33:09.276023+0000","last_change":"2024-04-26T04:33:04.889451+0000","last_active":"2024-04-26T04:33:09.276023+0000","last_peered":"2024-04-26T04:33:09.276023+0000","last_clean":"2024-04-26T04:33:09.276023+0000","last_became_active":"2024-04-26T04:32:23.392585+0000","last_became_peered":"2024-04-26T04:32:23.392585+0000","last_unstale":"2024-04-26T04:33:09.276023+0000","last_undegraded":"2024-04-26T04:33:09.276023+0000","last_fullsized":"2024-04-26T04:33:09.276023+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:31.059557+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.47307905700000003,"stat_sum":{"num_bytes":3915776,"num_objects":5,"num_object_clones":2,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":9755,"num_write":68,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"3.c","version":"0'0","reported_seq":192,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-26T04:33:09.273981+0000","last_change":"2024-04-26T04:33:04.264585+0000","last_active":"2024-04-26T04:33:09.273981+0000","last_peered":"2024-04-26T04:33:09.273981+0000","last_clean":"2024-04-26T04:33:09.273981+0000","last_became_active":"2024-04-26T04:32:23.577814+0000","last_became_peered":"2024-04-26T04:32:23.577814+0000","last_unstale":"2024-04-26T04:33:09.273981+0000","last_undegraded":"2024-04-26T04:33:09.273981+0000","last_fullsized":"2024-04-26T04:33:09.273981+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:31.141226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012010549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"3.b","version":"76'65","reported_seq":303,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-26T04:33:12.607875+0000","last_change":"2024-04-26T04:33:11.684436+0000","last_active":"2024-04-26T04:33:12.607875+0000","last_peered":"2024-04-26T04:33:12.607875+0000","last_clean":"2024-04-26T04:33:12.607875+0000","last_became_active":"2024-04-26T04:32:23.220443+0000","last_became_peered":"2024-04-26T04:32:23.220443+0000","last_unstale":"2024-04-26T04:33:12.607875+0000","last_undegraded":"2024-04-26T04:33:12.607875+0000","last_fullsized":"2024-04-26T04:33:12.607875+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:42.538308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.50446953500000002,"stat_sum":{"num_bytes":9093120,"num_objects":7,"num_object_clones":4,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":32806,"num_write":55,"num_write_kb":11024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"e","length":"1"},{"start":"15","length":"1"}]},{"pgid":"3.a","version":"68'41","reported_seq":242,"reported_epoch":73,"state":"remapped+peering","last_fresh":"2024-04-26T04:33:08.262957+0000","last_change":"2024-04-26T04:33:09.270060+0000","last_active":"2024-04-26T04:33:08.262957+0000","last_peered":"2024-04-26T04:33:07.718276+0000","last_clean":"2024-04-26T04:33:07.718276+0000","last_became_active":"2024-04-26T04:32:23.310795+0000","last_became_peered":"2024-04-26T04:32:23.310795+0000","last_unstale":"2024-04-26T04:33:08.262957+0000","last_undegraded":"2024-04-26T04:33:08.262957+0000","last_fullsized":"2024-04-26T04:33:08.262957+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093880899999999998,"stat_sum":{"num_bytes":3309568,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":9223,"num_write":18,"num_write_kb":5040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,13,6,9,8],"acting":[3,1,7,2147483647,6,2147483647,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6,7,8,9,11,13,14],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"3.9","version":"69'12","reported_seq":217,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-26T04:33:10.287320+0000","last_change":"2024-04-26T04:33:04.690826+0000","last_active":"2024-04-26T04:33:10.287320+0000","last_peered":"2024-04-26T04:33:10.287320+0000","last_clean":"2024-04-26T04:33:10.287320+0000","last_became_active":"2024-04-26T04:32:23.476584+0000","last_became_peered":"2024-04-26T04:32:23.476584+0000","last_unstale":"2024-04-26T04:33:10.287320+0000","last_undegraded":"2024-04-26T04:33:10.287320+0000","last_fullsized":"2024-04-26T04:33:10.287320+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:47.957571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20485212699999999,"stat_sum":{"num_bytes":458752,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":15,"num_read_kb":4132,"num_write":23,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"3.8","version":"69'95","reported_seq":304,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-26T04:33:09.370833+0000","last_change":"2024-04-26T04:33:05.295712+0000","last_active":"2024-04-26T04:33:09.370833+0000","last_peered":"2024-04-26T04:33:09.370833+0000","last_clean":"2024-04-26T04:33:09.370833+0000","last_became_active":"2024-04-26T04:32:23.487565+0000","last_became_peered":"2024-04-26T04:32:23.487565+0000","last_unstale":"2024-04-26T04:33:09.370833+0000","last_undegraded":"2024-04-26T04:33:09.370833+0000","last_fullsized":"2024-04-26T04:33:09.370833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:30.212078+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.833921735,"stat_sum":{"num_bytes":10764288,"num_objects":7,"num_object_clones":1,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":45,"num_read_kb":15761,"num_write":81,"num_write_kb":11728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,12,1,14,15,6],"acting":[5,0,12,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"1.0","version":"32'157","reported_seq":269,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-26T04:33:09.368414+0000","last_change":"2024-04-26T04:32:56.378977+0000","last_active":"2024-04-26T04:33:09.368414+0000","last_peered":"2024-04-26T04:33:09.368414+0000","last_clean":"2024-04-26T04:33:09.368414+0000","last_became_active":"2024-04-26T04:32:56.378173+0000","last_became_peered":"2024-04-26T04:32:56.378173+0000","last_unstale":"2024-04-26T04:33:09.368414+0000","last_undegraded":"2024-04-26T04:33:09.368414+0000","last_fullsized":"2024-04-26T04:33:09.368414+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:32:08.365921+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:25.422212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,13],"acting":[3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":106,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-26T04:33:09.370940+0000","last_change":"2024-04-26T04:32:12.505010+0000","last_active":"2024-04-26T04:33:09.370940+0000","last_peered":"2024-04-26T04:33:09.370940+0000","last_clean":"2024-04-26T04:33:09.370940+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:33:09.370940+0000","last_undegraded":"2024-04-26T04:33:09.370940+0000","last_fullsized":"2024-04-26T04:33:09.370940+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.374592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"44'12","reported_seq":167,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-26T04:33:12.531587+0000","last_change":"2024-04-26T04:33:11.389564+0000","last_active":"2024-04-26T04:33:12.531587+0000","last_peered":"2024-04-26T04:33:12.531587+0000","last_clean":"2024-04-26T04:33:12.531587+0000","last_became_active":"2024-04-26T04:32:56.531093+0000","last_became_peered":"2024-04-26T04:32:56.531093+0000","last_unstale":"2024-04-26T04:33:12.531587+0000","last_undegraded":"2024-04-26T04:33:12.531587+0000","last_fullsized":"2024-04-26T04:33:12.531587+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:23.777783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.105348159,"stat_sum":{"num_bytes":360448,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":22,"num_read_kb":4231,"num_write":8,"num_write_kb":1584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,10,3,6,15,1,11],"acting":[13,10,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[{"start":"e","length":"1"},{"start":"15","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":105,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-26T04:33:09.268383+0000","last_change":"2024-04-26T04:32:12.603501+0000","last_active":"2024-04-26T04:33:09.268383+0000","last_peered":"2024-04-26T04:33:09.268383+0000","last_clean":"2024-04-26T04:33:09.268383+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:33:09.268383+0000","last_undegraded":"2024-04-26T04:33:09.268383+0000","last_fullsized":"2024-04-26T04:33:09.268383+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:12.945207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"69'86","reported_seq":307,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-26T04:33:12.395105+0000","last_change":"2024-04-26T04:33:11.281823+0000","last_active":"2024-04-26T04:33:12.395105+0000","last_peered":"2024-04-26T04:33:12.395105+0000","last_clean":"2024-04-26T04:33:12.395105+0000","last_became_active":"2024-04-26T04:32:23.399115+0000","last_became_peered":"2024-04-26T04:32:23.399115+0000","last_unstale":"2024-04-26T04:33:12.395105+0000","last_undegraded":"2024-04-26T04:33:12.395105+0000","last_fullsized":"2024-04-26T04:33:12.395105+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:34.596189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.101678878,"stat_sum":{"num_bytes":12976128,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":44563,"num_write":53,"num_write_kb":15456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,12,13],"acting":[4,11,9,5,10,12,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"e","length":"1"},{"start":"15","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":37,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-26T04:33:09.370871+0000","last_change":"2024-04-26T04:32:56.277333+0000","last_active":"2024-04-26T04:33:09.370871+0000","last_peered":"2024-04-26T04:33:09.370871+0000","last_clean":"2024-04-26T04:33:09.370871+0000","last_became_active":"2024-04-26T04:32:56.276685+0000","last_became_peered":"2024-04-26T04:32:56.276685+0000","last_unstale":"2024-04-26T04:33:09.370871+0000","last_undegraded":"2024-04-26T04:33:09.370871+0000","last_fullsized":"2024-04-26T04:33:09.370871+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.299653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,13],"acting":[5,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"69'17","reported_seq":219,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-26T04:33:10.287291+0000","last_change":"2024-04-26T04:33:04.892513+0000","last_active":"2024-04-26T04:33:10.287291+0000","last_peered":"2024-04-26T04:33:10.287291+0000","last_clean":"2024-04-26T04:33:10.287291+0000","last_became_active":"2024-04-26T04:32:23.386975+0000","last_became_peered":"2024-04-26T04:32:23.386975+0000","last_unstale":"2024-04-26T04:33:10.287291+0000","last_undegraded":"2024-04-26T04:33:10.287291+0000","last_fullsized":"2024-04-26T04:33:10.287291+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.243949+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.40647535099999998,"stat_sum":{"num_bytes":1654784,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":4851,"num_write":10,"num_write_kb":1744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,11,15,1,5,6,4],"acting":[14,11,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"2.2","version":"0'0","reported_seq":105,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-26T04:33:09.272389+0000","last_change":"2024-04-26T04:32:12.798464+0000","last_active":"2024-04-26T04:33:09.272389+0000","last_peered":"2024-04-26T04:33:09.272389+0000","last_clean":"2024-04-26T04:33:09.272389+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:33:09.272389+0000","last_undegraded":"2024-04-26T04:33:09.272389+0000","last_fullsized":"2024-04-26T04:33:09.272389+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.684262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":192,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-26T04:33:09.375827+0000","last_change":"2024-04-26T04:33:04.366572+0000","last_active":"2024-04-26T04:33:09.375827+0000","last_peered":"2024-04-26T04:33:09.375827+0000","last_clean":"2024-04-26T04:33:09.375827+0000","last_became_active":"2024-04-26T04:32:23.387136+0000","last_became_peered":"2024-04-26T04:32:23.387136+0000","last_unstale":"2024-04-26T04:33:09.375827+0000","last_undegraded":"2024-04-26T04:33:09.375827+0000","last_fullsized":"2024-04-26T04:33:09.375827+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:46.076203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012675239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,6,5,3],"acting":[13,7,4,15,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"b","length":"1"},{"start":"e","length":"2"}]},{"pgid":"3.5","version":"77'86","reported_seq":308,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-26T04:33:12.484256+0000","last_change":"2024-04-26T04:33:11.383627+0000","last_active":"2024-04-26T04:33:12.484256+0000","last_peered":"2024-04-26T04:33:12.484256+0000","last_clean":"2024-04-26T04:33:12.484256+0000","last_became_active":"2024-04-26T04:32:23.295127+0000","last_became_peered":"2024-04-26T04:32:23.295127+0000","last_unstale":"2024-04-26T04:33:12.484256+0000","last_undegraded":"2024-04-26T04:33:12.484256+0000","last_fullsized":"2024-04-26T04:33:12.484256+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:50.410115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20800666700000001,"stat_sum":{"num_bytes":9371648,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":40,"num_read_kb":16527,"num_write":83,"num_write_kb":10016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"e","length":"1"},{"start":"15","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":110,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-26T04:33:12.395644+0000","last_change":"2024-04-26T04:32:13.108823+0000","last_active":"2024-04-26T04:33:12.395644+0000","last_peered":"2024-04-26T04:33:12.395644+0000","last_clean":"2024-04-26T04:33:12.395644+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:33:12.395644+0000","last_undegraded":"2024-04-26T04:33:12.395644+0000","last_fullsized":"2024-04-26T04:33:12.395644+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:20.545098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"77'86","reported_seq":304,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-26T04:33:12.495011+0000","last_change":"2024-04-26T04:33:11.235140+0000","last_active":"2024-04-26T04:33:12.495011+0000","last_peered":"2024-04-26T04:33:12.495011+0000","last_clean":"2024-04-26T04:33:12.495011+0000","last_became_active":"2024-04-26T04:32:23.290645+0000","last_became_peered":"2024-04-26T04:32:23.290645+0000","last_unstale":"2024-04-26T04:33:12.495011+0000","last_undegraded":"2024-04-26T04:33:12.495011+0000","last_fullsized":"2024-04-26T04:33:12.495011+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.747656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081104899999999999,"stat_sum":{"num_bytes":8699904,"num_objects":8,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":37,"num_read_kb":12397,"num_write":40,"num_write_kb":10880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,4,1,0,14,13],"acting":[8,9,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"e","length":"1"},{"start":"15","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-26T04:33:10.378003+0000","last_change":"2024-04-26T04:32:12.504400+0000","last_active":"2024-04-26T04:33:10.378003+0000","last_peered":"2024-04-26T04:33:10.378003+0000","last_clean":"2024-04-26T04:33:10.378003+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:33:10.378003+0000","last_undegraded":"2024-04-26T04:33:10.378003+0000","last_fullsized":"2024-04-26T04:33:10.378003+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.381954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"76'62","reported_seq":296,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-26T04:33:12.604440+0000","last_change":"2024-04-26T04:33:11.492309+0000","last_active":"2024-04-26T04:33:12.604440+0000","last_peered":"2024-04-26T04:33:12.604440+0000","last_clean":"2024-04-26T04:33:12.604440+0000","last_became_active":"2024-04-26T04:32:23.477983+0000","last_became_peered":"2024-04-26T04:32:23.477983+0000","last_unstale":"2024-04-26T04:33:12.604440+0000","last_undegraded":"2024-04-26T04:33:12.604440+0000","last_fullsized":"2024-04-26T04:33:12.604440+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:39.202723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.31122851600000001,"stat_sum":{"num_bytes":5603328,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":42,"num_read_kb":27144,"num_write":31,"num_write_kb":11056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"e","length":"1"},{"start":"15","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":110,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-26T04:33:12.394737+0000","last_change":"2024-04-26T04:32:12.697838+0000","last_active":"2024-04-26T04:33:12.394737+0000","last_peered":"2024-04-26T04:33:12.394737+0000","last_clean":"2024-04-26T04:33:12.394737+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:33:12.394737+0000","last_undegraded":"2024-04-26T04:33:12.394737+0000","last_fullsized":"2024-04-26T04:33:12.394737+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.961816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"70'58","reported_seq":273,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-26T04:33:12.288408+0000","last_change":"2024-04-26T04:33:11.385429+0000","last_active":"2024-04-26T04:33:12.288408+0000","last_peered":"2024-04-26T04:33:12.288408+0000","last_clean":"2024-04-26T04:33:12.288408+0000","last_became_active":"2024-04-26T04:32:23.383105+0000","last_became_peered":"2024-04-26T04:32:23.383105+0000","last_unstale":"2024-04-26T04:33:12.288408+0000","last_undegraded":"2024-04-26T04:33:12.288408+0000","last_fullsized":"2024-04-26T04:33:12.288408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:29.062413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10181891699999999,"stat_sum":{"num_bytes":10158080,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":10140,"num_write":72,"num_write_kb":9920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"e","length":"1"},{"start":"15","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-26T04:33:12.299543+0000","last_change":"2024-04-26T04:32:12.803651+0000","last_active":"2024-04-26T04:33:12.299543+0000","last_peered":"2024-04-26T04:33:12.299543+0000","last_clean":"2024-04-26T04:33:12.299543+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:33:12.299543+0000","last_undegraded":"2024-04-26T04:33:12.299543+0000","last_fullsized":"2024-04-26T04:33:12.299543+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:26.614652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":92831744,"num_objects":75,"num_object_clones":29,"num_object_copies":525,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":3,"num_read":464,"num_read_kb":223072,"num_write":672,"num_write_kb":116816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163569664,"data_stored":206344192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":826,"ondisk_log_size":826,"up":112,"acting":112,"num_store_stats":15},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1536000,"data_stored":1525344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247824,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46304,"kb_used_data":15124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325536,"statfs":{"total":96636764160,"available":96589348864,"internally_reserved":0,"allocated":15486976,"data_stored":19497527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247825,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41472,"kb_used_data":10292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330368,"statfs":{"total":96636764160,"available":96594296832,"internally_reserved":0,"allocated":10539008,"data_stored":10592823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247824,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46992,"kb_used_data":15812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324848,"statfs":{"total":96636764160,"available":96588644352,"internally_reserved":0,"allocated":16191488,"data_stored":20530263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313232,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48624,"kb_used_data":17436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323216,"statfs":{"total":96636764160,"available":96586973184,"internally_reserved":0,"allocated":17854464,"data_stored":21970363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313232,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40552,"kb_used_data":13536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331288,"statfs":{"total":96636764160,"available":96595238912,"internally_reserved":0,"allocated":13860864,"data_stored":16229603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,16,3,0,6],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313232,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39768,"kb_used_data":12748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94332072,"statfs":{"total":96636764160,"available":96596041728,"internally_reserved":0,"allocated":13053952,"data_stored":14994875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313232,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34652,"kb_used_data":7632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337188,"statfs":{"total":96636764160,"available":96601280512,"internally_reserved":0,"allocated":7815168,"data_stored":8854971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,3,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313232,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36092,"kb_used_data":9072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335748,"statfs":{"total":96636764160,"available":96599805952,"internally_reserved":0,"allocated":9289728,"data_stored":11951547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":98784247825,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280527,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30520,"kb_used_data":3492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341320,"statfs":{"total":96636764160,"available":96605511680,"internally_reserved":0,"allocated":3575808,"data_stored":3205858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280527,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45120,"kb_used_data":9772,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326720,"statfs":{"total":96636764160,"available":96590561280,"internally_reserved":0,"allocated":10006528,"data_stored":11630786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247824,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47256,"kb_used_data":16072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324584,"statfs":{"total":96636764160,"available":96588374016,"internally_reserved":0,"allocated":16457728,"data_stored":19721033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39684,"kb_used_data":8504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332156,"statfs":{"total":96636764160,"available":96596127744,"internally_reserved":0,"allocated":8708096,"data_stored":9404983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247823,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43512,"kb_used_data":12332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328328,"statfs":{"total":96636764160,"available":96592207872,"internally_reserved":0,"allocated":12627968,"data_stored":15352375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280527,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45060,"kb_used_data":9712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326780,"statfs":{"total":96636764160,"available":96590622720,"internally_reserved":0,"allocated":9945088,"data_stored":12024002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247825,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44632,"kb_used_data":13452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327208,"statfs":{"total":96636764160,"available":96591060992,"internally_reserved":0,"allocated":13774848,"data_stored":17637943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16805888,"data_stored":21544960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15392768,"data_stored":19116032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2011136,"data_stored":2093056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12828672,"data_stored":15814656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7749632,"data_stored":8900608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":11968512,"data_stored":14475264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8912896,"data_stored":12943360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12193792,"data_stored":14897152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":14438400,"data_stored":18903040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":6717440,"data_stored":8228864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":9531392,"data_stored":11751424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":12726272,"data_stored":17043456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":14610432,"data_stored":19308544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":9490432,"data_stored":9998336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":8192000,"data_stored":11325440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:33:13.614 INFO:tasks.ceph.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-04-26T04:33:13.629 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:13.629 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:13.630 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:33:13.630 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:13.643 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T04:33:13.700 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:13.697+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:13.801 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:13.796+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:13.802 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:13.802 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:13.803 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:33:13.803 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:33:13.803 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:13.817 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T04:33:13.901 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:13.900+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:13.985 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:13.985 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:33:13.985 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:33:13.985 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:33:13.985 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.985 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:22 3.0s0 3:0061323f:::smithi17829308-8:head [append 0~131072,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:13.985 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.206381+0000", 2024-04-26T04:33:13.985 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 48.779268706000003, 2024-04-26T04:33:13.986 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30553245699999998, 2024-04-26T04:33:13.986 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.986 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.986 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:13.987 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:33:13.987 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:33:13.987 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:13.987 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 22 2024-04-26T04:33:13.987 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.987 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.987 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.987 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.987 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206381+0000", 2024-04-26T04:33:13.987 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.987 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.987 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.987 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.988 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206381+0000", 2024-04-26T04:33:13.988 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.988 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.988 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.988 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206385+0000", 2024-04-26T04:33:13.988 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3299999999999999e-06 2024-04-26T04:33:13.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.988 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.988 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.988 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206772+0000", 2024-04-26T04:33:13.988 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000387673 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206797+0000", 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4156e-05 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206811+0000", 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4087000000000001e-05 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206881+0000", 2024-04-26T04:33:13.989 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9948000000000002e-05 2024-04-26T04:33:13.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.990 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.990 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:13.990 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206974+0000", 2024-04-26T04:33:13.990 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.3047000000000005e-05 2024-04-26T04:33:13.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.990 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.990 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.990 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207449+0000", 2024-04-26T04:33:13.990 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00047526900000000001 2024-04-26T04:33:13.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.990 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208764+0000", 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001315395 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511723+0000", 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.302958377 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511914+0000", 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000191175 2024-04-26T04:33:13.991 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.992 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.992 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.992 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.992 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.992 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:23 3.0s0 3:0061323f:::smithi17829308-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:13.992 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.206854+0000", 2024-04-26T04:33:13.992 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 48.778796190000001, 2024-04-26T04:33:13.992 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.407632722, 2024-04-26T04:33:13.992 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.992 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 23 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206854+0000", 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.993 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206854+0000", 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206855+0000", 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.133e-06 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206875+0000", 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9945999999999998e-05 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.994 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.995 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:13.995 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206889+0000", 2024-04-26T04:33:13.995 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3997e-05 2024-04-26T04:33:13.995 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.995 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.995 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:13.995 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206899+0000", 2024-04-26T04:33:13.995 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8490000000000001e-06 2024-04-26T04:33:13.995 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.995 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.995 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:13.995 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207769+0000", 2024-04-26T04:33:13.995 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00087023199999999997 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207895+0000", 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012591300000000001 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208201+0000", 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030561 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.209436+0000", 2024-04-26T04:33:13.996 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012354969999999999 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.614327+0000", 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.40489055000000002 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.614487+0000", 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015999499999999999 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:13.997 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:24 3.0s0 3:0061323f:::smithi17829308-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.206918+0000", 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 48.778732398000002, 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.612165556, 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 24 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.998 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206918+0000", 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206918+0000", 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206919+0000", 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5000000000000001e-07 2024-04-26T04:33:13.999 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206923+0000", 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0330000000000002e-06 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206934+0000", 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0866999999999999e-05 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206946+0000", 2024-04-26T04:33:14.000 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2947e-05 2024-04-26T04:33:14.001 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.001 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.001 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.001 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208374+0000", 2024-04-26T04:33:14.001 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014272930000000001 2024-04-26T04:33:14.001 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.001 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.001 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:14.001 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208613+0000", 2024-04-26T04:33:14.001 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00023916200000000001 2024-04-26T04:33:14.001 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.001 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.001 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.002 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.615561+0000", 2024-04-26T04:33:14.002 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.40694828199999999 2024-04-26T04:33:14.002 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.002 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.002 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:14.002 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.615825+0000", 2024-04-26T04:33:14.002 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00026369700000000002 2024-04-26T04:33:14.002 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.002 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.002 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.002 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.819083+0000", 2024-04-26T04:33:14.003 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20325832499999999 2024-04-26T04:33:14.003 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.003 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.003 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.003 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.003 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.003 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=0, reqid=client.4670.0:1, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:14.003 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:33:14.003 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 48.682803384000003, 2024-04-26T04:33:14.003 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.208159126, 2024-04-26T04:33:14.003 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.003 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.004 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:33:14.005 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8299999999999998e-07 2024-04-26T04:33:14.005 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.005 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.005 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.005 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302898+0000", 2024-04-26T04:33:14.005 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1421000000000001e-05 2024-04-26T04:33:14.005 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.005 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.005 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.005 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302912+0000", 2024-04-26T04:33:14.005 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4075e-05 2024-04-26T04:33:14.005 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.005 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.006 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.006 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302928+0000", 2024-04-26T04:33:14.006 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5333999999999999e-05 2024-04-26T04:33:14.006 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.006 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.006 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.006 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403201+0000", 2024-04-26T04:33:14.006 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100273162 2024-04-26T04:33:14.006 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.006 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.006 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.006 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403219+0000", 2024-04-26T04:33:14.007 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7526000000000001e-05 2024-04-26T04:33:14.007 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.007 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.007 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.007 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.510967+0000", 2024-04-26T04:33:14.007 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107748329 2024-04-26T04:33:14.007 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.007 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.007 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.007 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511006+0000", 2024-04-26T04:33:14.007 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8995999999999997e-05 2024-04-26T04:33:14.007 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.007 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=1, reqid=client.4670.0:2, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 48.682700306999998, 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.208096686, 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.008 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.82e-07 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.009 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302987+0000", 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6507999999999997e-05 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302996+0000", 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4900000000000006e-06 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303002+0000", 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3800000000000002e-06 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.010 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403906+0000", 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100904445 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403919+0000", 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.259e-05 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511012+0000", 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107093034 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.011 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511046+0000", 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4756999999999999e-05 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=2, reqid=client.4670.0:13, at_version=32'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.303011+0000", 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 48.682638842999999, 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20833385600000001, 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.012 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303011+0000", 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303011+0000", 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303012+0000", 2024-04-26T04:33:14.013 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4700000000000002e-07 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303033+0000", 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1123000000000001e-05 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303040+0000", 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5340000000000002e-06 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.014 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303044+0000", 2024-04-26T04:33:14.015 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1620000000000001e-06 2024-04-26T04:33:14.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.015 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.015 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.015 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.404280+0000", 2024-04-26T04:33:14.015 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101235978 2024-04-26T04:33:14.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.015 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.015 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.015 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.404303+0000", 2024-04-26T04:33:14.015 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2446000000000002e-05 2024-04-26T04:33:14.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.016 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.016 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.016 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511267+0000", 2024-04-26T04:33:14.016 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.1069645 2024-04-26T04:33:14.016 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.016 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.016 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.016 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511345+0000", 2024-04-26T04:33:14.016 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7766e-05 2024-04-26T04:33:14.016 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.016 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.016 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 32/30 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.305887+0000", 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 48.679763285, 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20516263700000001, 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305887+0000", 2024-04-26T04:33:14.017 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305887+0000", 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305888+0000", 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.108e-06 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.018 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305895+0000", 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2099999999999996e-06 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305914+0000", 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9164000000000002e-05 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305925+0000", 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1156e-05 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.019 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.510931+0000", 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20500514 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511049+0000", 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00011885899999999999 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 32/30 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-04-26T04:33:14.020 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.306725+0000", 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 48.678924918999996, 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20445080600000001, 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306725+0000", 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306725+0000", 2024-04-26T04:33:14.021 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.022 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.022 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.022 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.022 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306726+0000", 2024-04-26T04:33:14.022 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2500000000000004e-07 2024-04-26T04:33:14.022 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.022 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.022 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.022 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306731+0000", 2024-04-26T04:33:14.022 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9250000000000003e-06 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306745+0000", 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4367e-05 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306753+0000", 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9699999999999999e-06 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.023 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511144+0000", 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20439093 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511176+0000", 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1689000000000001e-05 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:280 3.0s0 3:0061323f:::smithi17829308-8:head [delete] snapc 2=[2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.865956+0000", 2024-04-26T04:33:14.024 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 43.119693951999999, 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20424141400000001, 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 280 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865956+0000", 2024-04-26T04:33:14.025 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.026 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.026 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.026 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865956+0000", 2024-04-26T04:33:14.026 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.026 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.026 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865959+0000", 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0570000000000001e-06 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865971+0000", 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1576000000000001e-05 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866003+0000", 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2543999999999999e-05 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866020+0000", 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6433e-05 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866157+0000", 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013767200000000001 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866327+0000", 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016995199999999999 2024-04-26T04:33:14.028 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.029 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.029 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.029 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.867476+0000", 2024-04-26T04:33:14.029 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0011486700000000001 2024-04-26T04:33:14.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.029 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.029 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.029 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.869167+0000", 2024-04-26T04:33:14.029 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016912450000000001 2024-04-26T04:33:14.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.029 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.029 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:14.029 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.070080+0000", 2024-04-26T04:33:14.030 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200912916 2024-04-26T04:33:14.030 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.030 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.030 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.030 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.070198+0000", 2024-04-26T04:33:14.030 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000117349 2024-04-26T04:33:14.030 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.030 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.030 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.030 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.030 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.030 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds2 40/30 ECSubWrite(tid=33, reqid=client.4670.0:332, at_version=0'0, trim_to=0'0, roll_forward_to=40'19))", 2024-04-26T04:33:14.030 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:35.023904+0000", 2024-04-26T04:33:14.031 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 38.961745729999997, 2024-04-26T04:33:14.031 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201717813, 2024-04-26T04:33:14.031 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.031 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.031 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.031 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.031 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.031 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.031 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023904+0000", 2024-04-26T04:33:14.031 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.031 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023904+0000", 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023906+0000", 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.387e-06 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023925+0000", 2024-04-26T04:33:14.032 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9059000000000002e-05 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023952+0000", 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7455999999999999e-05 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023968+0000", 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.526e-05 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.033 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.224267+0000", 2024-04-26T04:33:14.034 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200299636 2024-04-26T04:33:14.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.034 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.034 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.034 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.224282+0000", 2024-04-26T04:33:14.034 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5007e-05 2024-04-26T04:33:14.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.034 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.034 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.034 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.225547+0000", 2024-04-26T04:33:14.034 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012650719999999999 2024-04-26T04:33:14.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.034 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.225622+0000", 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4936000000000002e-05 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.4s5 45/30 ECSubWrite(tid=34, reqid=client.4670.0:377, at_version=45'23, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:39.634866+0000", 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 34.350784101000002, 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20147746799999999, 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.035 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634866+0000", 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634866+0000", 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.036 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634866+0000", 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8599999999999999e-07 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634876+0000", 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8649999999999996e-06 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.634884+0000", 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7060000000000004e-06 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.037 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.038 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.734989+0000", 2024-04-26T04:33:14.038 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100104926 2024-04-26T04:33:14.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.038 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.038 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.038 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.835213+0000", 2024-04-26T04:33:14.038 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100223672 2024-04-26T04:33:14.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.038 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.038 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.038 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.835223+0000", 2024-04-26T04:33:14.038 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0570999999999999e-05 2024-04-26T04:33:14.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.836270+0000", 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0010470220000000001 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:39.836343+0000", 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3319999999999996e-05 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds2 52/30 ECSubWrite(tid=52, reqid=osd.7.0:52, at_version=52'34, trim_to=0'0, roll_forward_to=50'32))", 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:46.171817+0000", 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 27.813833015, 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.199021011, 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.171817+0000", 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.040 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.171817+0000", 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.171817+0000", 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8000000000000001e-07 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.171854+0000", 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6297000000000002e-05 2024-04-26T04:33:14.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.171880+0000", 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5769999999999999e-05 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.171897+0000", 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7646e-05 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.369183+0000", 2024-04-26T04:33:14.042 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.197285603 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.369219+0000", 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6699000000000002e-05 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.370728+0000", 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0015088269999999999 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.043 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:46.370838+0000", 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000109789 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.as4 62/30 ECSubWrite(tid=49, reqid=client.4670.0:645, at_version=0'0, trim_to=0'0, roll_forward_to=62'36))", 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:57.714367+0000", 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 16.271283294, 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201555649, 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.044 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.714367+0000", 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.714367+0000", 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.714369+0000", 2024-04-26T04:33:14.045 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3199999999999998e-06 2024-04-26T04:33:14.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.046 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.046 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.046 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.714392+0000", 2024-04-26T04:33:14.046 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3249000000000001e-05 2024-04-26T04:33:14.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.046 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.046 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.046 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.714437+0000", 2024-04-26T04:33:14.046 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4203999999999999e-05 2024-04-26T04:33:14.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.046 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.046 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.714460+0000", 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3163000000000002e-05 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.914817+0000", 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20035745399999999 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.914835+0000", 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7839e-05 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.047 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.915888+0000", 2024-04-26T04:33:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0010526979999999999 2024-04-26T04:33:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.915922+0000", 2024-04-26T04:33:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4721999999999997e-05 2024-04-26T04:33:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.048 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:722 3.9s0 3:979b0971:::smithi17829308-28:head [stat,delete] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-26T04:33:14.049 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:01.995106+0000", 2024-04-26T04:33:14.049 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 11.990544431, 2024-04-26T04:33:14.049 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30568222699999997, 2024-04-26T04:33:14.049 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.049 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.049 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:14.049 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:33:14.049 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:33:14.049 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.049 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 722 2024-04-26T04:33:14.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.049 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.050 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.050 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.050 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995106+0000", 2024-04-26T04:33:14.050 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.050 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.050 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.050 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995106+0000", 2024-04-26T04:33:14.050 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.050 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.050 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.051 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995108+0000", 2024-04-26T04:33:14.051 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6070000000000002e-06 2024-04-26T04:33:14.051 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.051 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.051 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.051 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995121+0000", 2024-04-26T04:33:14.051 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.272e-05 2024-04-26T04:33:14.051 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.051 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.051 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.051 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995155+0000", 2024-04-26T04:33:14.051 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3639000000000001e-05 2024-04-26T04:33:14.051 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995171+0000", 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6427e-05 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995367+0000", 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00019574499999999999 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995540+0000", 2024-04-26T04:33:14.052 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017338799999999999 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.997167+0000", 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016268700000000001 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.999322+0000", 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021550810000000001 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:14.053 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.300652+0000", 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.301329601 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.300788+0000", 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013614899999999999 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.as4 66/30 ECSubWrite(tid=53, reqid=client.4670.0:702, at_version=0'0, trim_to=0'0, roll_forward_to=66'39))", 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:02.315960+0000", 2024-04-26T04:33:14.054 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 11.669690436, 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201565358, 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.315960+0000", 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.315960+0000", 2024-04-26T04:33:14.055 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.315962+0000", 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1639999999999999e-06 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.315988+0000", 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5868999999999999e-05 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.056 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.316053+0000", 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4832999999999994e-05 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.416155+0000", 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100102605 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.516408+0000", 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10025274000000001 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.057 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.516425+0000", 2024-04-26T04:33:14.058 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7283e-05 2024-04-26T04:33:14.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.058 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.058 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.058 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.517433+0000", 2024-04-26T04:33:14.058 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0010080779999999999 2024-04-26T04:33:14.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.058 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.058 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.058 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.517525+0000", 2024-04-26T04:33:14.058 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.1786000000000005e-05 2024-04-26T04:33:14.058 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.058 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 67/30 ECSubWriteReply(tid=41, last_complete=66'8, committed=1, applied=1))", 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:02.402204+0000", 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 11.583446263000001, 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.300648372, 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.402204+0000", 2024-04-26T04:33:14.059 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.402204+0000", 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.402206+0000", 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.703e-06 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.060 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.402215+0000", 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6500000000000008e-06 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.402244+0000", 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9156999999999999e-05 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.502387+0000", 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014247 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.061 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.062 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.702789+0000", 2024-04-26T04:33:14.062 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200401736 2024-04-26T04:33:14.062 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.062 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.062 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.062 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.702852+0000", 2024-04-26T04:33:14.062 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.3656000000000002e-05 2024-04-26T04:33:14.062 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.062 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.062 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.062 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.062 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.062 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 68/30 ECSubWrite(tid=107, reqid=osd.5.0:107, at_version=68'81, trim_to=0'0, roll_forward_to=61'79))", 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.433160+0000", 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.552490192, 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20302263600000001, 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.433160+0000", 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.063 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.433160+0000", 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.433160+0000", 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2599999999999998e-07 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.433208+0000", 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7320000000000001e-05 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.064 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.433232+0000", 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4620000000000001e-05 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.433259+0000", 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.656e-05 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.533592+0000", 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10033310300000001 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.065 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.066 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.533607+0000", 2024-04-26T04:33:14.066 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5330000000000001e-05 2024-04-26T04:33:14.066 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.066 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.066 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.066 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.636100+0000", 2024-04-26T04:33:14.066 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102492578 2024-04-26T04:33:14.066 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.066 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.066 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.066 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.636183+0000", 2024-04-26T04:33:14.066 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2699000000000002e-05 2024-04-26T04:33:14.066 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 68/30 ECSubWrite(tid=108, reqid=osd.5.0:108, at_version=68'83, trim_to=0'0, roll_forward_to=61'79))", 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.434841+0000", 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.550809376, 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20139731999999999, 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.067 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.434841+0000", 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.434841+0000", 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.434842+0000", 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.42e-06 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.068 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.434877+0000", 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4388999999999999e-05 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.434895+0000", 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8773000000000001e-05 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.434909+0000", 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3667e-05 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.069 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.534444+0000", 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.099535483999999994 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.534460+0000", 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5133000000000001e-05 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.636193+0000", 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10173328399999999 2024-04-26T04:33:14.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.636238+0000", 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5170000000000003e-05 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds2 68/30 ECSubWrite(tid=72, reqid=osd.7.0:72, at_version=68'50, trim_to=0'0, roll_forward_to=67'46))", 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.705256+0000", 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.280394324, 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20317829500000001, 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.071 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705256+0000", 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705256+0000", 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.072 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.073 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705257+0000", 2024-04-26T04:33:14.073 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.424e-06 2024-04-26T04:33:14.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.073 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.073 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.073 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705304+0000", 2024-04-26T04:33:14.073 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6879000000000003e-05 2024-04-26T04:33:14.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.073 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.073 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.073 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705339+0000", 2024-04-26T04:33:14.073 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.472e-05 2024-04-26T04:33:14.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705364+0000", 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5360000000000001e-05 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.905749+0000", 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20038445199999999 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.905766+0000", 2024-04-26T04:33:14.074 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.698e-05 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.908334+0000", 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0025682439999999999 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.908434+0000", 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010023599999999999 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 68/30 ECSubWrite(tid=110, reqid=osd.5.0:110, at_version=68'85, trim_to=0'0, roll_forward_to=68'83))", 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.838699+0000", 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.146950711000001, 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201654738, 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.838699+0000", 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.077 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.077 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:14.077 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.838699+0000", 2024-04-26T04:33:14.077 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:14.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.077 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.077 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:14.077 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.838701+0000", 2024-04-26T04:33:14.077 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9630000000000001e-06 2024-04-26T04:33:14.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.077 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.077 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:14.077 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.838743+0000", 2024-04-26T04:33:14.078 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1579000000000003e-05 2024-04-26T04:33:14.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.078 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.078 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:14.078 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.838772+0000", 2024-04-26T04:33:14.078 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9507000000000001e-05 2024-04-26T04:33:14.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.078 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.078 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.078 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.938855+0000", 2024-04-26T04:33:14.078 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10008270900000001 2024-04-26T04:33:14.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.078 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:14.079 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.039044+0000", 2024-04-26T04:33:14.079 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100188612 2024-04-26T04:33:14.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.079 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.079 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.039062+0000", 2024-04-26T04:33:14.079 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8085999999999999e-05 2024-04-26T04:33:14.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.079 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.079 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.040266+0000", 2024-04-26T04:33:14.079 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012044759999999999 2024-04-26T04:33:14.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:14.080 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:14.080 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:14.080 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.040354+0000", 2024-04-26T04:33:14.080 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.7806000000000006e-05 2024-04-26T04:33:14.080 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.080 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.080 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.080 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:14.080 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:14.080 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:14.082 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T04:33:14.082 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:14.080+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:14.182 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:14.180+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:14.216 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:14.216 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:14.216 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.216 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:900 3.as0 3:527650bd:::smithi17829308-27:head [read 0~2686976,checksum in=4b,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-04-26T04:33:14.216 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:13.516193+0000", 2024-04-26T04:33:14.216 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.69994795099999996, 2024-04-26T04:33:14.216 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.69999628700000005, 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "waiting for missing object", 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 900 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516193+0000", 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.217 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.218 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.218 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.218 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516193+0000", 2024-04-26T04:33:14.218 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.218 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.218 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.218 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.218 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516194+0000", 2024-04-26T04:33:14.218 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.68e-06 2024-04-26T04:33:14.218 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.218 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.218 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.218 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516200+0000", 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9070000000000004e-06 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516213+0000", 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2991e-05 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516219+0000", 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8100000000000003e-06 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.219 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516352+0000", 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000132638 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for missing object", 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.517825+0000", 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0014731670000000001 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:901 3.as0 3:527650bd:::smithi17829308-27:head [read 0~2686976,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-04-26T04:33:14.220 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:13.516231+0000", 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.69990967699999995, 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.70008347299999996, 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "waiting for missing object", 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 901 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516231+0000", 2024-04-26T04:33:14.221 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.222 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.222 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.222 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.222 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516231+0000", 2024-04-26T04:33:14.222 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.222 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.222 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.222 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.222 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516231+0000", 2024-04-26T04:33:14.222 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7599999999999998e-07 2024-04-26T04:33:14.222 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.222 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.222 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516234+0000", 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4310000000000001e-06 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516238+0000", 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.4660000000000001e-06 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516241+0000", 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0769999999999999e-06 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.223 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.517899+0000", 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016572609999999999 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for missing object", 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.518098+0000", 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000199271 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:902 3.as0 3:527650bd:::smithi17829308-27:head [read 0~2686976,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:13.516342+0000", 2024-04-26T04:33:14.224 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.69979897800000002, 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.70008363500000004, 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "waiting for missing object", 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 902 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516342+0000", 2024-04-26T04:33:14.225 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516342+0000", 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516343+0000", 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.37e-06 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.226 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516352+0000", 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1430000000000004e-06 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516376+0000", 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3572999999999999e-05 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516390+0000", 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4259e-05 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.227 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.518152+0000", 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001761987 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for missing object", 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.518329+0000", 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000176849 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpReadReply(3.as0 78/74 ECSubReadReply(tid=73, attrs_read=0))", 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:14.213213+0000", 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.0029279810000000001, 2024-04-26T04:33:14.228 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.003312181, 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.213213+0000", 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.213213+0000", 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.229 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.213215+0000", 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7969999999999999e-06 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.213497+0000", 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00028258599999999998 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.213524+0000", 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7285999999999999e-05 2024-04-26T04:33:14.230 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.213539+0000", 2024-04-26T04:33:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4192999999999999e-05 2024-04-26T04:33:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-04-26T04:33:14.231 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 4 2024-04-26T04:33:14.231 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:14.232 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T04:33:14.283 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:14.281+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:14.297 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-04-26T04:33:14.304 INFO:tasks.rados.rados.0.smithi178.stdout:352: done (2 left) 2024-04-26T04:33:14.304 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 5 current snap is 23 2024-04-26T04:33:14.304 INFO:tasks.rados.rados.0.smithi178.stdout:353: seq_num 138 ranges {0=770048,770048=540672,1310720=753664,2064384=262144} 2024-04-26T04:33:14.316 INFO:tasks.rados.rados.0.smithi178.stdout:353: writing smithi17829308-5 from 0 to 770048 tid 1 2024-04-26T04:33:14.325 INFO:tasks.rados.rados.0.smithi178.stdout:353: writing smithi17829308-5 from 770048 to 1310720 tid 2 2024-04-26T04:33:14.336 INFO:tasks.rados.rados.0.smithi178.stdout:353: writing smithi17829308-5 from 1310720 to 2064384 tid 3 2024-04-26T04:33:14.340 INFO:tasks.rados.rados.0.smithi178.stdout:353: writing smithi17829308-5 from 2064384 to 2326528 tid 4 2024-04-26T04:33:14.340 INFO:tasks.rados.rados.0.smithi178.stdout:354: delete oid 37 current snap is 23 2024-04-26T04:33:14.340 INFO:tasks.rados.rados.0.smithi178.stdout:341: finishing write tid 6 to smithi17829308-11 2024-04-26T04:33:14.340 INFO:tasks.rados.rados.0.smithi178.stdout:341: oid 11 updating version 0 to 68 2024-04-26T04:33:14.340 INFO:tasks.rados.rados.0.smithi178.stdout:341: oid 11 updating version 68 to 70 2024-04-26T04:33:14.340 INFO:tasks.rados.rados.0.smithi178.stdout:341: oid 11 version 70 is already newer than 67 2024-04-26T04:33:14.341 INFO:tasks.rados.rados.0.smithi178.stdout:341: oid 11 version 70 is already newer than 66 2024-04-26T04:33:14.341 INFO:tasks.rados.rados.0.smithi178.stdout:341: oid 11 version 70 is already newer than 69 2024-04-26T04:33:14.341 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 11 v 70 (ObjNum 137 snap 22 seq_num 137) dirty exists 2024-04-26T04:33:14.341 INFO:tasks.rados.rados.0.smithi178.stdout:341: left oid 11 (ObjNum 137 snap 22 seq_num 137) 2024-04-26T04:33:14.359 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:14.359 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:14.359 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:33:14.359 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:33:14.359 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:14.374 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T04:33:14.385 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:33:14.377+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:33:14.485 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:14.484+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:14.504 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:14.504 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:33:14.504 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:33:14.504 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:14.505 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.505 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:265 3.as0 3:51a91c23:::smithi17829308-18:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:14.505 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.817865+0000", 2024-04-26T04:33:14.505 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 44.682952981, 2024-04-26T04:33:14.505 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50897067299999998, 2024-04-26T04:33:14.505 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.505 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.505 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:14.505 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.505 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.505 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.505 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 265 2024-04-26T04:33:14.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.506 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.506 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.506 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.506 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817865+0000", 2024-04-26T04:33:14.506 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.506 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.506 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.506 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817865+0000", 2024-04-26T04:33:14.506 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.507 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.507 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.507 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817867+0000", 2024-04-26T04:33:14.507 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0999999999999998e-06 2024-04-26T04:33:14.507 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.507 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.507 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.507 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817876+0000", 2024-04-26T04:33:14.507 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8060000000000001e-06 2024-04-26T04:33:14.507 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.507 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.507 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.507 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817899+0000", 2024-04-26T04:33:14.508 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2716000000000001e-05 2024-04-26T04:33:14.508 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.508 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.508 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.508 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817910+0000", 2024-04-26T04:33:14.508 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1579999999999999e-05 2024-04-26T04:33:14.508 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.508 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.508 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.508 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.822433+0000", 2024-04-26T04:33:14.508 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0045222869999999998 2024-04-26T04:33:14.508 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.509 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.509 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.509 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.822747+0000", 2024-04-26T04:33:14.509 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00031401299999999998 2024-04-26T04:33:14.509 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.509 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.509 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.509 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.823790+0000", 2024-04-26T04:33:14.509 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00104357 2024-04-26T04:33:14.509 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.510 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.510 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.510 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.924740+0000", 2024-04-26T04:33:14.510 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100949605 2024-04-26T04:33:14.510 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.510 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.510 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:14.510 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.326718+0000", 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40197800900000002 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.326836+0000", 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011798699999999999 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:328 3.as0 3:527650bd:::smithi17829308-27:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.408149+0000", 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 40.092669649000001, 2024-04-26T04:33:14.511 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.409867282, 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 328 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408149+0000", 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.512 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408149+0000", 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408152+0000", 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.8870000000000002e-06 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408172+0000", 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.912e-05 2024-04-26T04:33:14.513 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.514 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.514 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.514 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408216+0000", 2024-04-26T04:33:14.514 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.4370999999999999e-05 2024-04-26T04:33:14.514 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.514 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.514 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.514 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408238+0000", 2024-04-26T04:33:14.514 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1820000000000001e-05 2024-04-26T04:33:14.514 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.514 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.514 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.514 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408355+0000", 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000116882 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408364+0000", 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.4099999999999997e-06 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.412080+0000", 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.003715566 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.515 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.516 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.412245+0000", 2024-04-26T04:33:14.516 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000165914 2024-04-26T04:33:14.516 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.516 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.516 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.516 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.413482+0000", 2024-04-26T04:33:14.516 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0012361690000000001 2024-04-26T04:33:14.516 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.516 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.516 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.516 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616957+0000", 2024-04-26T04:33:14.516 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20347567 2024-04-26T04:33:14.516 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.817777+0000", 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20081958 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.818016+0000", 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000238893 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.517 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:329 3.as0 3:527650bd:::smithi17829308-27:head [rollback 2] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.408276+0000", 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 40.092541744999998, 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61054785099999997, 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 329 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.518 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.519 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.519 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408276+0000", 2024-04-26T04:33:14.519 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.519 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.519 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.519 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.519 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408276+0000", 2024-04-26T04:33:14.519 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.519 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.519 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.519 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.519 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408277+0000", 2024-04-26T04:33:14.519 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.7999999999999993e-07 2024-04-26T04:33:14.520 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.520 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.520 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.520 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408284+0000", 2024-04-26T04:33:14.520 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.0389999999999999e-06 2024-04-26T04:33:14.520 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.520 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.520 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.520 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408311+0000", 2024-04-26T04:33:14.521 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6452999999999999e-05 2024-04-26T04:33:14.521 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.521 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.521 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.521 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408324+0000", 2024-04-26T04:33:14.521 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3259e-05 2024-04-26T04:33:14.521 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.521 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.521 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.521 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408439+0000", 2024-04-26T04:33:14.521 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000114789 2024-04-26T04:33:14.522 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.522 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.522 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:14.522 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408444+0000", 2024-04-26T04:33:14.522 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.3700000000000003e-06 2024-04-26T04:33:14.522 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.522 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.522 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.522 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.414140+0000", 2024-04-26T04:33:14.522 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0056952110000000004 2024-04-26T04:33:14.522 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.522 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.522 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.414263+0000", 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012392699999999999 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.415016+0000", 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00075301600000000002 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616959+0000", 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20194278800000001 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.523 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018750+0000", 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40179092999999999 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018824+0000", 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.4089000000000002e-05 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.524 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:330 3.as0 3:527650bd:::smithi17829308-27:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:14.525 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.408341+0000", 2024-04-26T04:33:14.525 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 40.092476910999999, 2024-04-26T04:33:14.525 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61070586699999996, 2024-04-26T04:33:14.525 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.525 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.525 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:14.525 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.525 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.525 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.525 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 330 2024-04-26T04:33:14.525 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.525 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.525 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.526 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.526 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408341+0000", 2024-04-26T04:33:14.526 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.526 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.526 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.526 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.526 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408341+0000", 2024-04-26T04:33:14.526 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.526 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.526 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.526 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.526 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408342+0000", 2024-04-26T04:33:14.527 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.6700000000000003e-07 2024-04-26T04:33:14.527 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.527 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.527 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.527 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408346+0000", 2024-04-26T04:33:14.527 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7299999999999999e-06 2024-04-26T04:33:14.527 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.527 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.527 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.527 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408357+0000", 2024-04-26T04:33:14.527 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1276e-05 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408371+0000", 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3763e-05 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408472+0000", 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010149600000000001 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:14.528 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408477+0000", 2024-04-26T04:33:14.529 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5029999999999999e-06 2024-04-26T04:33:14.529 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.529 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.529 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.529 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.515978+0000", 2024-04-26T04:33:14.529 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.107501546 2024-04-26T04:33:14.529 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.529 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.529 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.529 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.516082+0000", 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010331 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.516573+0000", 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000491316 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.617004+0000", 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100431009 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.530 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:14.531 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018996+0000", 2024-04-26T04:33:14.531 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40199249199999998 2024-04-26T04:33:14.531 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.531 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.531 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.531 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.019047+0000", 2024-04-26T04:33:14.531 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0658999999999997e-05 2024-04-26T04:33:14.531 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.531 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.531 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.532 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.532 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.532 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:33:14.532 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.520051+0000", 2024-04-26T04:33:14.532 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 39.980766785999997, 2024-04-26T04:33:14.532 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49885426900000002, 2024-04-26T04:33:14.532 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.532 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.532 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.532 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.532 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.532 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.532 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520051+0000", 2024-04-26T04:33:14.533 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.533 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.533 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.533 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.533 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520051+0000", 2024-04-26T04:33:14.533 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.533 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.533 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.533 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.533 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520052+0000", 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.5000000000000001e-07 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520062+0000", 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9799999999999993e-06 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520079+0000", 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6518999999999999e-05 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.534 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.535 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520087+0000", 2024-04-26T04:33:14.535 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.5329999999999994e-06 2024-04-26T04:33:14.535 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.535 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.535 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.535 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018885+0000", 2024-04-26T04:33:14.535 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49879782299999997 2024-04-26T04:33:14.535 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.535 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.535 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.535 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018906+0000", 2024-04-26T04:33:14.536 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0564000000000002e-05 2024-04-26T04:33:14.536 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.536 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.536 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.536 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.536 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.536 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:33:14.536 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.522047+0000", 2024-04-26T04:33:14.536 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 39.978771101, 2024-04-26T04:33:14.536 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49659926700000001, 2024-04-26T04:33:14.537 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.537 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.537 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.537 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.537 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.537 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.537 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522047+0000", 2024-04-26T04:33:14.537 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.537 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.537 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.537 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.538 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522047+0000", 2024-04-26T04:33:14.538 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.538 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.538 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.538 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.538 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522048+0000", 2024-04-26T04:33:14.538 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1200000000000001e-07 2024-04-26T04:33:14.538 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.538 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.538 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.538 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522054+0000", 2024-04-26T04:33:14.538 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.049e-06 2024-04-26T04:33:14.538 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522070+0000", 2024-04-26T04:33:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5492999999999999e-05 2024-04-26T04:33:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522079+0000", 2024-04-26T04:33:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1679999999999993e-06 2024-04-26T04:33:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018623+0000", 2024-04-26T04:33:14.540 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49654389700000001 2024-04-26T04:33:14.540 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.540 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.540 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.540 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018646+0000", 2024-04-26T04:33:14.540 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3748e-05 2024-04-26T04:33:14.540 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.540 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.540 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.540 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.540 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.540 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=27, last_complete=40'19, committed=1, applied=1))", 2024-04-26T04:33:14.540 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.616517+0000", 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 39.884301256000001, 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40232084899999998, 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616517+0000", 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616517+0000", 2024-04-26T04:33:14.541 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.542 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.542 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.542 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.542 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616518+0000", 2024-04-26T04:33:14.542 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.0999999999999997e-07 2024-04-26T04:33:14.542 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.542 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.542 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.542 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616523+0000", 2024-04-26T04:33:14.542 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0490000000000003e-06 2024-04-26T04:33:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616539+0000", 2024-04-26T04:33:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6555999999999999e-05 2024-04-26T04:33:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616549+0000", 2024-04-26T04:33:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8269999999999993e-06 2024-04-26T04:33:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.543 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.544 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018674+0000", 2024-04-26T04:33:14.544 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40212457600000001 2024-04-26T04:33:14.544 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.544 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.544 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.544 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018838+0000", 2024-04-26T04:33:14.544 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000164031 2024-04-26T04:33:14.544 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.544 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.544 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.544 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.544 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.616951+0000", 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 39.883867487000003, 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.402569169, 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616951+0000", 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.545 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.546 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616951+0000", 2024-04-26T04:33:14.546 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.546 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.546 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.546 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.546 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616951+0000", 2024-04-26T04:33:14.546 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.8700000000000005e-07 2024-04-26T04:33:14.546 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.546 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616957+0000", 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.3589999999999999e-06 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616974+0000", 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7278999999999998e-05 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616983+0000", 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9500000000000007e-06 2024-04-26T04:33:14.547 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.548 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.548 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.548 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018945+0000", 2024-04-26T04:33:14.548 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40196159399999998 2024-04-26T04:33:14.548 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.548 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.548 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.548 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.019520+0000", 2024-04-26T04:33:14.548 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0005752 2024-04-26T04:33:14.548 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.548 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.548 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.548 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:333 3.as0 3:527650bd:::smithi17829308-27:head [copy-from ver 2] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.120049+0000", 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 39.380769366999999, 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.66333988200000005, 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 333 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.549 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120049+0000", 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120049+0000", 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120050+0000", 2024-04-26T04:33:14.550 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.536e-06 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120057+0000", 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6869999999999997e-06 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120074+0000", 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6787e-05 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.551 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120086+0000", 2024-04-26T04:33:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2665e-05 2024-04-26T04:33:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.220306+0000", 2024-04-26T04:33:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10021996900000001 2024-04-26T04:33:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.220485+0000", 2024-04-26T04:33:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017836800000000001 2024-04-26T04:33:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.577412+0000", 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.35692740299999998 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.580481+0000", 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.003068735 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.783160+0000", 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20267940200000001 2024-04-26T04:33:14.553 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.554 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.554 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.554 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.783389+0000", 2024-04-26T04:33:14.554 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022833 2024-04-26T04:33:14.554 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.554 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.554 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.554 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.554 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.554 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:334 3.as0 3:527650bd:::smithi17829308-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:14.554 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.120106+0000", 2024-04-26T04:33:14.554 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 39.380711884, 2024-04-26T04:33:14.554 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.459292215, 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 334 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120106+0000", 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.555 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120106+0000", 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120107+0000", 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.540000000000001e-07 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120111+0000", 2024-04-26T04:33:14.556 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.6160000000000002e-06 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120124+0000", 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2965e-05 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.220210+0000", 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100086047 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221197+0000", 2024-04-26T04:33:14.557 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00098664200000000003 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221256+0000", 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9734000000000001e-05 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.578868+0000", 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.35761166799999999 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.558 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579093+0000", 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022499099999999999 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579399+0000", 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00030559800000000002 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:620 3.as0 3:57b6ca82:::smithi17829308-14:b [read 0~229376,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:55.807425+0000", 2024-04-26T04:33:14.559 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 18.693393389000001, 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50556072699999999, 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 620 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807425+0000", 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.561 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.561 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807425+0000", 2024-04-26T04:33:14.561 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.561 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.561 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807429+0000", 2024-04-26T04:33:14.561 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.0459999999999999e-06 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807448+0000", 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9553000000000001e-05 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807498+0000", 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.9369000000000001e-05 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.562 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807524+0000", 2024-04-26T04:33:14.563 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5843e-05 2024-04-26T04:33:14.563 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.563 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.563 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907865+0000", 2024-04-26T04:33:14.563 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100341135 2024-04-26T04:33:14.563 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.563 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.563 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.908044+0000", 2024-04-26T04:33:14.563 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017972100000000001 2024-04-26T04:33:14.563 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.564 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.564 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.312986+0000", 2024-04-26T04:33:14.564 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40494106000000002 2024-04-26T04:33:14.564 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.564 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.564 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.564 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.564 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:621 3.as0 3:57b6ca82:::smithi17829308-14:b [read 0~229376,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:14.564 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:55.807584+0000", 2024-04-26T04:33:14.564 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 18.693234042, 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50635941100000004, 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 621 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807584+0000", 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.566 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.566 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807584+0000", 2024-04-26T04:33:14.566 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.566 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.566 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807586+0000", 2024-04-26T04:33:14.566 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.466e-06 2024-04-26T04:33:14.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.567 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.567 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807595+0000", 2024-04-26T04:33:14.567 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.064e-06 2024-04-26T04:33:14.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.567 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.567 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807616+0000", 2024-04-26T04:33:14.567 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1379e-05 2024-04-26T04:33:14.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.567 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.568 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907810+0000", 2024-04-26T04:33:14.568 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10019436800000001 2024-04-26T04:33:14.568 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.568 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.568 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.009320+0000", 2024-04-26T04:33:14.568 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101509213 2024-04-26T04:33:14.568 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.568 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.009472+0000", 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015200399999999999 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.313944+0000", 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30447191699999998 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:622 3.as0 3:57b6ca82:::smithi17829308-14:b [read 0~229376,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:14.569 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:55.907855+0000", 2024-04-26T04:33:14.570 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 18.592963263000001, 2024-04-26T04:33:14.570 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.806439982, 2024-04-26T04:33:14.570 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.570 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.570 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:14.570 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.570 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.570 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.570 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 622 2024-04-26T04:33:14.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.571 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.571 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.571 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907855+0000", 2024-04-26T04:33:14.571 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.571 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.571 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907855+0000", 2024-04-26T04:33:14.571 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907857+0000", 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2000000000000001e-06 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907869+0000", 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.19e-05 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907889+0000", 2024-04-26T04:33:14.572 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9643000000000001e-05 2024-04-26T04:33:14.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.573 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.573 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907901+0000", 2024-04-26T04:33:14.573 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2289999999999999e-05 2024-04-26T04:33:14.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.573 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.573 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.210837+0000", 2024-04-26T04:33:14.573 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30293618300000003 2024-04-26T04:33:14.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.574 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.574 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.211009+0000", 2024-04-26T04:33:14.574 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017150999999999999 2024-04-26T04:33:14.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.574 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.574 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.714295+0000", 2024-04-26T04:33:14.574 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50328625599999999 2024-04-26T04:33:14.574 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.574 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.574 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:645 3.as0 3:51a91c23:::smithi17829308-18:head [copy-from ver 49] snapc f=[f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:57.194710+0000", 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 17.306108571999999, 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.517523497, 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 645 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.575 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.576 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.576 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194710+0000", 2024-04-26T04:33:14.576 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.576 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.576 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194710+0000", 2024-04-26T04:33:14.576 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.577 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.577 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194714+0000", 2024-04-26T04:33:14.577 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.07e-06 2024-04-26T04:33:14.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.577 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.577 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194727+0000", 2024-04-26T04:33:14.577 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3593e-05 2024-04-26T04:33:14.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.577 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.577 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194762+0000", 2024-04-26T04:33:14.578 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.5086000000000003e-05 2024-04-26T04:33:14.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.578 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.578 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194782+0000", 2024-04-26T04:33:14.578 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9349000000000001e-05 2024-04-26T04:33:14.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.578 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.578 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194905+0000", 2024-04-26T04:33:14.578 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000123497 2024-04-26T04:33:14.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194923+0000", 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7377e-05 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.298801+0000", 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10387803 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.298959+0000", 2024-04-26T04:33:14.579 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000158484 2024-04-26T04:33:14.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.580 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.580 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.509187+0000", 2024-04-26T04:33:14.580 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21022799 2024-04-26T04:33:14.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.580 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.580 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.611520+0000", 2024-04-26T04:33:14.580 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.102332901 2024-04-26T04:33:14.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.580 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:14.581 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.712044+0000", 2024-04-26T04:33:14.581 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10052366 2024-04-26T04:33:14.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.581 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.581 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.712233+0000", 2024-04-26T04:33:14.581 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018945999999999999 2024-04-26T04:33:14.581 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.581 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.581 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.582 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:646 3.as0 3:51a91c23:::smithi17829308-18:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:33:14.582 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:57.194821+0000", 2024-04-26T04:33:14.582 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 17.305996897, 2024-04-26T04:33:14.582 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.416436949, 2024-04-26T04:33:14.582 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.582 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.582 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:14.582 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.582 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 646 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194821+0000", 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194821+0000", 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194822+0000", 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.04e-06 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194827+0000", 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.8760000000000004e-06 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194841+0000", 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3446e-05 2024-04-26T04:33:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194850+0000", 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8740000000000001e-06 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.295147+0000", 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100297979 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.295158+0000", 2024-04-26T04:33:14.585 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1e-05 2024-04-26T04:33:14.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.586 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.586 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.499442+0000", 2024-04-26T04:33:14.586 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20428347899999999 2024-04-26T04:33:14.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.586 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:14.586 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.499538+0000", 2024-04-26T04:33:14.586 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6527999999999995e-05 2024-04-26T04:33:14.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.586 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.610922+0000", 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.111383597 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.611079+0000", 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0001571 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.611258+0000", 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017903 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.587 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:702 3.as0 3:527650bd:::smithi17829308-27:head [copy-from ver 44] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-26T04:33:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:01.580554+0000", 2024-04-26T04:33:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 12.920263678, 2024-04-26T04:33:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.73333148100000001, 2024-04-26T04:33:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.588 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 702 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580554+0000", 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580554+0000", 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.589 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580556+0000", 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9910000000000001e-06 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580570+0000", 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3798000000000001e-05 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580599+0000", 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8591000000000001e-05 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580615+0000", 2024-04-26T04:33:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6515e-05 2024-04-26T04:33:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580817+0000", 2024-04-26T04:33:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00020131200000000001 2024-04-26T04:33:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.581112+0000", 2024-04-26T04:33:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00029485500000000001 2024-04-26T04:33:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.109360+0000", 2024-04-26T04:33:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.52824810099999997 2024-04-26T04:33:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.114011+0000", 2024-04-26T04:33:14.592 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0046516190000000001 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.313710+0000", 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.19969835499999999 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.313886+0000", 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017634399999999999 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:703 3.as0 3:527650bd:::smithi17829308-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e66)", 2024-04-26T04:33:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:01.580660+0000", 2024-04-26T04:33:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 12.920157877999999, 2024-04-26T04:33:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.53310025599999999, 2024-04-26T04:33:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 703 2024-04-26T04:33:14.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580660+0000", 2024-04-26T04:33:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580660+0000", 2024-04-26T04:33:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.595 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580661+0000", 2024-04-26T04:33:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.95e-07 2024-04-26T04:33:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580666+0000", 2024-04-26T04:33:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0919999999999997e-06 2024-04-26T04:33:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580684+0000", 2024-04-26T04:33:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7578e-05 2024-04-26T04:33:14.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580694+0000", 2024-04-26T04:33:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0339999999999999e-05 2024-04-26T04:33:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.781619+0000", 2024-04-26T04:33:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20092432399999999 2024-04-26T04:33:14.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.781728+0000", 2024-04-26T04:33:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010893200000000001 2024-04-26T04:33:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.113439+0000", 2024-04-26T04:33:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.33171105400000001 2024-04-26T04:33:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.598 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.113592+0000", 2024-04-26T04:33:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000153731 2024-04-26T04:33:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.113761+0000", 2024-04-26T04:33:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016820999999999999 2024-04-26T04:33:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:825 3.as0 3:51a91c23:::smithi17829308-18:head [read 0~393216,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.293542+0000", 2024-04-26T04:33:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 7.2072764789999999, 2024-04-26T04:33:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.42480815300000002, 2024-04-26T04:33:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 825 2024-04-26T04:33:14.600 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293542+0000", 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293542+0000", 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.601 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293542+0000", 2024-04-26T04:33:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.2500000000000001e-07 2024-04-26T04:33:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293545+0000", 2024-04-26T04:33:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3529999999999999e-06 2024-04-26T04:33:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293552+0000", 2024-04-26T04:33:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.5570000000000001e-06 2024-04-26T04:33:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293559+0000", 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1420000000000004e-06 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293779+0000", 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022003100000000001 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293797+0000", 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8102999999999999e-05 2024-04-26T04:33:14.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.398220+0000", 2024-04-26T04:33:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10442280600000001 2024-04-26T04:33:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.398489+0000", 2024-04-26T04:33:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026918299999999999 2024-04-26T04:33:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.718350+0000", 2024-04-26T04:33:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.31986055299999999 2024-04-26T04:33:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:836 3.as0 3:57b6ca82:::smithi17829308-14:head [rollback 14] snapc 16=[16,15,14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-04-26T04:33:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:09.269087+0000", 2024-04-26T04:33:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.2317310380000004, 2024-04-26T04:33:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0499148229999999, 2024-04-26T04:33:14.605 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 836 2024-04-26T04:33:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.606 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269087+0000", 2024-04-26T04:33:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269087+0000", 2024-04-26T04:33:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269089+0000", 2024-04-26T04:33:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4470000000000001e-06 2024-04-26T04:33:14.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269099+0000", 2024-04-26T04:33:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0657e-05 2024-04-26T04:33:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269120+0000", 2024-04-26T04:33:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1222000000000001e-05 2024-04-26T04:33:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.608 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269137+0000", 2024-04-26T04:33:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6143000000000001e-05 2024-04-26T04:33:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.675959+0000", 2024-04-26T04:33:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40682288 2024-04-26T04:33:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for peered", 2024-04-26T04:33:14.609 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.675965+0000", 2024-04-26T04:33:14.610 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4559999999999999e-06 2024-04-26T04:33:14.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.610 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.610 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:11.102281+0000", 2024-04-26T04:33:14.610 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4263155759999999 2024-04-26T04:33:14.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.610 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for missing object", 2024-04-26T04:33:14.610 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:11.102551+0000", 2024-04-26T04:33:14.610 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000270396 2024-04-26T04:33:14.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.109278+0000", 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.006727003 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.109514+0000", 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00023647699999999999 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.110889+0000", 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0013741630000000001 2024-04-26T04:33:14.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.114040+0000", 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0031512649999999999 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.318345+0000", 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20430549100000001 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.319002+0000", 2024-04-26T04:33:14.612 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000656647 2024-04-26T04:33:14.613 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.613 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.613 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.613 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:837 3.as0 3:57b6ca82:::smithi17829308-14:head [append 0~0] snapc 16=[16,15,14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-04-26T04:33:14.613 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:09.269156+0000", 2024-04-26T04:33:14.613 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 5.2316618449999996, 2024-04-26T04:33:14.613 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1507874779999998, 2024-04-26T04:33:14.613 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.613 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.613 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 837 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269156+0000", 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.614 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269156+0000", 2024-04-26T04:33:14.615 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.615 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.615 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269158+0000", 2024-04-26T04:33:14.615 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2160000000000001e-06 2024-04-26T04:33:14.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.615 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.615 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269163+0000", 2024-04-26T04:33:14.615 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.5770000000000001e-06 2024-04-26T04:33:14.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269181+0000", 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7629999999999999e-05 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269198+0000", 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7376000000000001e-05 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.676017+0000", 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40681856900000002 2024-04-26T04:33:14.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.617 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for peered", 2024-04-26T04:33:14.617 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.676018+0000", 2024-04-26T04:33:14.617 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.59e-06 2024-04-26T04:33:14.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.617 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.617 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:11.102957+0000", 2024-04-26T04:33:14.617 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4269388190000001 2024-04-26T04:33:14.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.617 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for missing object", 2024-04-26T04:33:14.618 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:11.103123+0000", 2024-04-26T04:33:14.618 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000165909 2024-04-26T04:33:14.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.618 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:14.618 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.111738+0000", 2024-04-26T04:33:14.618 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0086147130000001 2024-04-26T04:33:14.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.618 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:14.618 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.111986+0000", 2024-04-26T04:33:14.618 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000248371 2024-04-26T04:33:14.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.113210+0000", 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001223607 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.215036+0000", 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101826497 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.419214+0000", 2024-04-26T04:33:14.619 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.204177839 2024-04-26T04:33:14.620 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.620 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:14.620 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.419944+0000", 2024-04-26T04:33:14.620 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00072976499999999995 2024-04-26T04:33:14.620 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.620 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.620 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.620 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.620 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.620 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:14.622 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T04:33:14.623 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:14.618+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:14.657 INFO:tasks.rados.rados.0.smithi178.stdout:341: done (3 left) 2024-04-26T04:33:14.657 INFO:tasks.rados.rados.0.smithi178.stdout:354: done (2 left) 2024-04-26T04:33:14.657 INFO:tasks.rados.rados.0.smithi178.stdout:355: rollback oid 7 current snap is 23 2024-04-26T04:33:14.657 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 7 to 20 2024-04-26T04:33:14.657 INFO:tasks.rados.rados.0.smithi178.stdout:356: rmattr oid 16 current snap is 23 2024-04-26T04:33:14.658 INFO:tasks.rados.rados.0.smithi178.stdout:357: snap_remove snap 15 2024-04-26T04:33:14.723 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:14.722+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:14.789 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:14.789 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:14.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.789 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 79/30 ECSubWrite(tid=130, reqid=osd.15.0:130, at_version=79'99, trim_to=0'0, roll_forward_to=79'96))", 2024-04-26T04:33:14.789 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:14.765951+0000", 2024-04-26T04:33:14.789 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.023200149999999999, 2024-04-26T04:33:14.790 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.023237612000000001, 2024-04-26T04:33:14.790 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.790 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.790 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:14.790 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.790 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.790 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.790 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.765951+0000", 2024-04-26T04:33:14.790 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.790 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.791 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.791 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.791 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.765951+0000", 2024-04-26T04:33:14.791 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.791 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.791 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.791 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.765953+0000", 2024-04-26T04:33:14.791 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3030000000000001e-06 2024-04-26T04:33:14.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.791 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.791 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.792 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.766009+0000", 2024-04-26T04:33:14.792 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.6702e-05 2024-04-26T04:33:14.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.792 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.792 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.766034+0000", 2024-04-26T04:33:14.792 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4289000000000001e-05 2024-04-26T04:33:14.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.792 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.766056+0000", 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2557000000000001e-05 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.bs5 79/30 ECSubWrite(tid=344, reqid=osd.4.0:342, at_version=0'0, trim_to=0'0, roll_forward_to=79'75))", 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:14.766185+0000", 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.022967016, 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.023091143000000001, 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:14.793 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.766185+0000", 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.766185+0000", 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.766186+0000", 2024-04-26T04:33:14.794 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1999999999999998e-07 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.766206+0000", 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.052e-05 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.766232+0000", 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6069000000000001e-05 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:14.795 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.766252+0000", 2024-04-26T04:33:14.796 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0123e-05 2024-04-26T04:33:14.796 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.796 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:14.796 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.796 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:14.796 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-04-26T04:33:14.796 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 2 2024-04-26T04:33:14.796 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:14.804 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T04:33:14.824 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:14.817+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:14.925 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:14.925+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:14.968 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:14.968 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:14.968 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:33:14.968 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:33:14.969 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:14.983 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T04:33:15.024 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T04:33:15.027 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:15.026+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:15.105 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:15.106 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:33:15.106 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:33:15.106 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:15.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.106 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:381 3.ds0 3:b7d7af40:::smithi17829308-48:5 [read 0~802816,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:33:15.106 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.431434+0000", 2024-04-26T04:33:15.106 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 35.672494235000002, 2024-04-26T04:33:15.106 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40271601499999998, 2024-04-26T04:33:15.106 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.107 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.107 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:15.107 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.107 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.107 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.107 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 381 2024-04-26T04:33:15.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.107 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.107 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.107 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431434+0000", 2024-04-26T04:33:15.107 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.108 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.108 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431434+0000", 2024-04-26T04:33:15.108 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.108 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.108 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431436+0000", 2024-04-26T04:33:15.108 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3140000000000002e-06 2024-04-26T04:33:15.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.109 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.109 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431447+0000", 2024-04-26T04:33:15.109 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0655e-05 2024-04-26T04:33:15.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.109 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.109 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431471+0000", 2024-04-26T04:33:15.109 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4470000000000001e-05 2024-04-26T04:33:15.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.109 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.110 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431488+0000", 2024-04-26T04:33:15.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6129000000000001e-05 2024-04-26T04:33:15.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.110 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.110 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.494163+0000", 2024-04-26T04:33:15.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.062675393999999995 2024-04-26T04:33:15.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.110 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:15.110 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.494174+0000", 2024-04-26T04:33:15.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.114e-05 2024-04-26T04:33:15.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.111 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.497530+0000", 2024-04-26T04:33:15.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0033555439999999998 2024-04-26T04:33:15.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.111 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.497684+0000", 2024-04-26T04:33:15.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015432600000000001 2024-04-26T04:33:15.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.112 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.112 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.834150+0000", 2024-04-26T04:33:15.112 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.33646604299999999 2024-04-26T04:33:15.112 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.112 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.112 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.112 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:382 3.ds0 3:b7d7af40:::smithi17829308-48:5 [read 0~802816,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:33:15.112 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:33:15.112 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 35.672415626999999, 2024-04-26T04:33:15.113 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50469882499999996, 2024-04-26T04:33:15.113 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.113 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.113 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:15.113 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.113 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.113 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.113 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 382 2024-04-26T04:33:15.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.113 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.113 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.113 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:33:15.114 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.114 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.114 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:33:15.114 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.115 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.115 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:33:15.115 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9800000000000003e-07 2024-04-26T04:33:15.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.115 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.115 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431521+0000", 2024-04-26T04:33:15.115 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7449999999999995e-06 2024-04-26T04:33:15.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.115 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.116 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431529+0000", 2024-04-26T04:33:15.116 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.3599999999999996e-06 2024-04-26T04:33:15.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.116 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.116 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531644+0000", 2024-04-26T04:33:15.116 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100114514 2024-04-26T04:33:15.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.116 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.116 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.699490+0000", 2024-04-26T04:33:15.116 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.16784555000000001 2024-04-26T04:33:15.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.117 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.117 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.699653+0000", 2024-04-26T04:33:15.117 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016372199999999999 2024-04-26T04:33:15.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.117 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.117 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.936211+0000", 2024-04-26T04:33:15.117 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23655823600000001 2024-04-26T04:33:15.117 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.117 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:383 3.ds0 3:b7d7af40:::smithi17829308-48:5 [read 0~802816,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.531687+0000", 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 35.572241093999999, 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.405754104, 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 383 2024-04-26T04:33:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531687+0000", 2024-04-26T04:33:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531687+0000", 2024-04-26T04:33:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531689+0000", 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.063e-06 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531702+0000", 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3156000000000001e-05 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531733+0000", 2024-04-26T04:33:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0335999999999999e-05 2024-04-26T04:33:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531766+0000", 2024-04-26T04:33:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3623e-05 2024-04-26T04:33:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.700194+0000", 2024-04-26T04:33:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.16842744400000001 2024-04-26T04:33:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.122 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.122 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.700343+0000", 2024-04-26T04:33:15.122 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000148763 2024-04-26T04:33:15.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.122 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.122 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.937441+0000", 2024-04-26T04:33:15.122 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23709871900000001 2024-04-26T04:33:15.122 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.122 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.122 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.123 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:424 3.ds0 3:b7d7af40:::smithi17829308-48:head [append 0~720896] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:33:15.123 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.921237+0000", 2024-04-26T04:33:15.123 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 30.182691643999998, 2024-04-26T04:33:15.123 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40632559499999998, 2024-04-26T04:33:15.123 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.123 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.123 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:15.123 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.123 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.123 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.123 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 424 2024-04-26T04:33:15.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.124 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.124 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.124 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.921237+0000", 2024-04-26T04:33:15.124 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.124 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.124 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.921237+0000", 2024-04-26T04:33:15.124 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.921239+0000", 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2900000000000001e-06 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922033+0000", 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00079434500000000001 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922059+0000", 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5925999999999999e-05 2024-04-26T04:33:15.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.126 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.126 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922072+0000", 2024-04-26T04:33:15.126 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2943e-05 2024-04-26T04:33:15.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.126 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.126 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922191+0000", 2024-04-26T04:33:15.126 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000118581 2024-04-26T04:33:15.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.126 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922362+0000", 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017091599999999999 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.924002+0000", 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016405359999999999 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.928906+0000", 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0049035720000000001 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.127 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.128 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:15.128 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327373+0000", 2024-04-26T04:33:15.128 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39846698000000003 2024-04-26T04:33:15.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.128 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.128 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.128 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327562+0000", 2024-04-26T04:33:15.128 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018950600000000001 2024-04-26T04:33:15.128 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.128 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.128 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.128 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:425 3.ds0 3:b7d7af40:::smithi17829308-48:head [append 0~49152] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.922098+0000", 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 30.181829799999999, 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.406108466, 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 425 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.129 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.130 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.130 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922098+0000", 2024-04-26T04:33:15.130 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.130 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.130 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.130 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.130 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922098+0000", 2024-04-26T04:33:15.130 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.130 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.130 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.130 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.130 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922100+0000", 2024-04-26T04:33:15.130 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2300000000000001e-06 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922189+0000", 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9623999999999994e-05 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922208+0000", 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8643000000000001e-05 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.131 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922219+0000", 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0699999999999999e-05 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.925187+0000", 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0029683930000000002 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.925371+0000", 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000184382 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:15.132 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.926426+0000", 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0010545190000000001 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.928909+0000", 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.002482981 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328000+0000", 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.399091539 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.133 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328207+0000", 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000206455 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:426 3.ds0 3:b7d7af40:::smithi17829308-48:head [setxattr _header (56) in=7b] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.922241+0000", 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 30.181687644, 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40635279400000002, 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.134 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 426 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922241+0000", 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922241+0000", 2024-04-26T04:33:15.135 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922242+0000", 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0470000000000001e-06 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922246+0000", 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5029999999999999e-06 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.136 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922257+0000", 2024-04-26T04:33:15.137 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.059e-05 2024-04-26T04:33:15.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.137 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.137 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.137 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922265+0000", 2024-04-26T04:33:15.137 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.8869999999999992e-06 2024-04-26T04:33:15.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.137 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.137 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.137 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.927201+0000", 2024-04-26T04:33:15.137 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0049363640000000004 2024-04-26T04:33:15.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.927353+0000", 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015164000000000001 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.928331+0000", 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00097882500000000001 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.029157+0000", 2024-04-26T04:33:15.138 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100825282 2024-04-26T04:33:15.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.139 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.139 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:15.139 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328408+0000", 2024-04-26T04:33:15.139 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29925099700000002 2024-04-26T04:33:15.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.139 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.139 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.139 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328593+0000", 2024-04-26T04:33:15.139 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018565899999999999 2024-04-26T04:33:15.139 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.139 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.139 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.140 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:427 3.ds0 3:b7d7af40:::smithi17829308-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:33:15.140 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.922276+0000", 2024-04-26T04:33:15.140 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 30.181652537000002, 2024-04-26T04:33:15.140 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.57447843799999998, 2024-04-26T04:33:15.140 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.140 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.140 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 427 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922276+0000", 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.141 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922276+0000", 2024-04-26T04:33:15.142 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.142 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.142 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922277+0000", 2024-04-26T04:33:15.142 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.7000000000000003e-07 2024-04-26T04:33:15.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.142 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.142 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922280+0000", 2024-04-26T04:33:15.143 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0589999999999998e-06 2024-04-26T04:33:15.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.143 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.143 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922293+0000", 2024-04-26T04:33:15.143 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3617e-05 2024-04-26T04:33:15.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.143 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.143 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922302+0000", 2024-04-26T04:33:15.143 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4300000000000006e-06 2024-04-26T04:33:15.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.144 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.144 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.929362+0000", 2024-04-26T04:33:15.144 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.007060439 2024-04-26T04:33:15.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.144 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.144 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:15.144 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.929534+0000", 2024-04-26T04:33:15.144 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017232900000000001 2024-04-26T04:33:15.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.144 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.144 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.144 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.329547+0000", 2024-04-26T04:33:15.145 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40001209799999998 2024-04-26T04:33:15.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.145 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.145 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.329786+0000", 2024-04-26T04:33:15.145 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00023956800000000001 2024-04-26T04:33:15.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.145 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.145 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.496754+0000", 2024-04-26T04:33:15.145 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.16696802799999999 2024-04-26T04:33:15.145 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.146 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.146 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.146 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.146 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 56/30 ECSubWrite(tid=60, reqid=client.4670.0:478, at_version=56'42, trim_to=0'0, roll_forward_to=50'39))", 2024-04-26T04:33:15.146 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.321773+0000", 2024-04-26T04:33:15.146 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 24.782155675999999, 2024-04-26T04:33:15.146 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50217973100000002, 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321773+0000", 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321773+0000", 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321773+0000", 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4399999999999999e-07 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321891+0000", 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011750499999999999 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321903+0000", 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.255e-05 2024-04-26T04:33:15.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.421985+0000", 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100081395 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.522172+0000", 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10018666 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.522181+0000", 2024-04-26T04:33:15.149 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8349999999999999e-06 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823898+0000", 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30171669200000001 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823952+0000", 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4249999999999997e-05 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.151 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 56/30 ECSubWrite(tid=61, reqid=client.4670.0:479, at_version=56'43, trim_to=0'0, roll_forward_to=50'39))", 2024-04-26T04:33:15.151 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.422026+0000", 2024-04-26T04:33:15.151 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 24.681902402999999, 2024-04-26T04:33:15.151 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40196675300000001, 2024-04-26T04:33:15.151 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.151 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.151 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:15.151 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.151 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.152 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.152 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422026+0000", 2024-04-26T04:33:15.152 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.152 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.152 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.152 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422026+0000", 2024-04-26T04:33:15.152 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.152 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.152 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.152 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422026+0000", 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.6599999999999996e-07 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422072+0000", 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5596000000000002e-05 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422086+0000", 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3577000000000001e-05 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.153 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422094+0000", 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.6319999999999997e-06 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.622832+0000", 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20073786099999999 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.622864+0000", 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1495000000000003e-05 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.154 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823965+0000", 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20110186299999999 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823993+0000", 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7163000000000001e-05 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.155 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 56/30 ECSubWrite(tid=62, reqid=client.4670.0:480, at_version=56'44, trim_to=0'0, roll_forward_to=50'39))", 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.422105+0000", 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 24.681822859, 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40191106599999998, 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422105+0000", 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.156 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422105+0000", 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422106+0000", 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1399999999999998e-07 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422115+0000", 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.5319999999999995e-06 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.157 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422130+0000", 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4474999999999999e-05 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422136+0000", 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7039999999999997e-06 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.623647+0000", 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201510879 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.158 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.623662+0000", 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5122e-05 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823997+0000", 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.200334135 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.824016+0000", 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9905e-05 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.159 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:635 3.ds0 3:b761f772:::smithi17829308-43:head [create,setxattr ]PaibOQQZPXlPQ_TVRVsq`]`lQnrU_adXdhR`ia_ (33) in=40b,setxattr mga[\\pjjlr_f^pjpoNm[SiQinb`p]Uljsr\\PZamg (1614) in=40b,setxattr \\\\nNdqV]P_aoSQ]T][SZi`\\`emqRalsThkWkp]_O (1998) in=40b,setxattr QPRVcO^PYksRnjf^RqtasbqiemoSYpeimc]_kQOo (1505) in=40b,setxattr i`dWfsdYVZUqQmWleQeniob`plYiVWUWSsnsd`\\a (1017) in=40b,setxattr Qn`WN^VrYNfSm_nWkOlO[bZ_[Zci__kgsaZeg[_R (1614) in=40b,setxattr \\]NdaY]dOTpk\\bgtmgifQtgWtaqfd]r`TbktPRjh (19016) in=40b,setxattr NlQZaoabhi (349) in=10b,setxattr QbZ^bqgnd]QnP\\ZjeRjaP[UfZgdm[V^sRaRpnZYY (1963) in=40b,setxattr `]YWO_jgNfUs_SQ`RaqUhXmNY[TtZZbaUj\\NPtY^ (1190) in=40b,setxattr dnfqsNhbe]^ml]ekS[ST`fnk_kpiTUa]Y]ges`rW (17824) in=40b,setxattr RgOUnWhYeYcgnikgiQQWNr_OWg_k`YSaN^XQOdPc (1437) in=40b,setxattr QUmNqZgRZOU]WaWWgl\\pnRhtilqS`cQaScmgUp_e (1438) in=40b,setxattr TfW^Smr_geRfti]efnPdr`iWQUikeXOp`StYqtQd (12280) in=40b,setxattr ZtZ (349) in=3b,setxattr NQYfjd\\ (349) in=7b,setxattr aNaTblXeObdSN[ahZn`j[mcbUaSjeXOnc[dZ^ZZo (910) in=40b,setxattr deb]iS_OUlbmWR_sP^er[TdaV`cSqnnthSotq]YZ (7928) in=40b,setxattr ^s_gVdo]kXUSXb]aqYjbUYbRmcoYksXSWQdVsh (349) in=38b,setxattr h^YTbW\\dli\\^dffQaXRNaZQVkZPONqhhabmhR_[s (1822) in=40b] snapc e=[e,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:56.110252+0000", 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 18.993675961000001, 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40734936900000002, 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.160 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 635 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110252+0000", 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110252+0000", 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.161 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110255+0000", 2024-04-26T04:33:15.162 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6560000000000001e-06 2024-04-26T04:33:15.162 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.162 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.162 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.162 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110399+0000", 2024-04-26T04:33:15.162 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000144098 2024-04-26T04:33:15.162 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.162 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.162 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.162 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110437+0000", 2024-04-26T04:33:15.162 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7867000000000002e-05 2024-04-26T04:33:15.162 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.162 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110447+0000", 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0183000000000001e-05 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110619+0000", 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017148900000000001 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110847+0000", 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000228199 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.163 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.164 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:15.164 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.112826+0000", 2024-04-26T04:33:15.164 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0019796530000000001 2024-04-26T04:33:15.164 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.164 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.164 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:15.164 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.115968+0000", 2024-04-26T04:33:15.164 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.003141657 2024-04-26T04:33:15.164 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.164 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.164 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:15.164 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.517378+0000", 2024-04-26T04:33:15.164 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.401409617 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.517602+0000", 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022395 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:740 3.ds0 3:bd5491ce:::smithi17829308-20:head [append 0~507904] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.364648+0000", 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 12.739280459, 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.72537579699999999, 2024-04-26T04:33:15.165 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 740 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.364648+0000", 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.166 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.364648+0000", 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.364652+0000", 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7100000000000001e-06 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.364987+0000", 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00033577699999999999 2024-04-26T04:33:15.167 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.365026+0000", 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.8359999999999999e-05 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.365044+0000", 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8578999999999999e-05 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.365197+0000", 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015245800000000001 2024-04-26T04:33:15.168 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.365211+0000", 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4303e-05 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.468736+0000", 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10352488 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.468920+0000", 2024-04-26T04:33:15.169 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018384299999999999 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.471244+0000", 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0023246579999999998 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.477358+0000", 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0061135470000000004 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:15.170 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.089886+0000", 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61252856499999997 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.090024+0000", 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000137117 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:741 3.ds0 3:bd5491ce:::smithi17829308-20:head [append 0~425984] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.368769+0000", 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 12.735159111, 2024-04-26T04:33:15.171 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.72154272200000003, 2024-04-26T04:33:15.172 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.172 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.172 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:15.172 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.172 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.172 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.172 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 741 2024-04-26T04:33:15.172 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.172 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.172 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.172 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.172 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.368769+0000", 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.368769+0000", 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.368772+0000", 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.503e-06 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.173 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.369295+0000", 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00052320399999999999 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.369324+0000", 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9260000000000001e-05 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.369339+0000", 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5079000000000001e-05 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.174 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.472521+0000", 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10318134900000001 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.472684+0000", 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016300099999999999 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.474045+0000", 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0013610339999999999 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.175 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.480049+0000", 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0060043520000000001 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.090206+0000", 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61015733800000005 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.090312+0000", 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000105602 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.176 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:742 3.ds0 3:bd5491ce:::smithi17829308-20:head [append 0~442368] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.372518+0000", 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 12.731410168, 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81824782500000004, 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.177 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 742 2024-04-26T04:33:15.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.178 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.178 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.178 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.178 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.372518+0000", 2024-04-26T04:33:15.178 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.178 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.178 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.178 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.372518+0000", 2024-04-26T04:33:15.178 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.178 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.372520+0000", 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1440000000000001e-06 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.373038+0000", 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00051735099999999999 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.373070+0000", 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2215999999999998e-05 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.179 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.373086+0000", 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5817000000000001e-05 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.474948+0000", 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101862724 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.475105+0000", 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015690300000000001 2024-04-26T04:33:15.180 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.476472+0000", 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0013667810000000001 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.480143+0000", 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0036706120000000002 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.190647+0000", 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71050477899999998 2024-04-26T04:33:15.181 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.190766+0000", 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000118498 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:743 3.ds0 3:bd5491ce:::smithi17829308-20:head [append 0~540672] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.376986+0000", 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 12.726942578999999, 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81437894700000002, 2024-04-26T04:33:15.182 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 743 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.376986+0000", 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.183 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.376986+0000", 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.376988+0000", 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2400000000000002e-06 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.377588+0000", 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00059970599999999998 2024-04-26T04:33:15.184 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.377621+0000", 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3402999999999999e-05 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.377637+0000", 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5489999999999999e-05 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.477443+0000", 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.099806487999999999 2024-04-26T04:33:15.185 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.477613+0000", 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017017399999999999 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.479104+0000", 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0014903480000000001 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.684294+0000", 2024-04-26T04:33:15.186 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20519051999999999 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.191246+0000", 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50695185700000001 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.191365+0000", 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000118721 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.187 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:744 3.ds0 3:bd5491ce:::smithi17829308-20:head [append 0~147456] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.378092+0000", 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 12.725836320999999, 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.813543248, 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 744 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.188 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378092+0000", 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378092+0000", 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378095+0000", 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.667e-06 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.189 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378258+0000", 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016352399999999999 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378291+0000", 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2601999999999997e-05 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378306+0000", 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5307000000000001e-05 2024-04-26T04:33:15.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.580438+0000", 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20213155699999999 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.580618+0000", 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018046699999999999 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.581709+0000", 2024-04-26T04:33:15.191 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001090466 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.684369+0000", 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.102660494 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.191539+0000", 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.507170122 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.191635+0000", 2024-04-26T04:33:15.192 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6042e-05 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:745 3.ds0 3:bd5491ce:::smithi17829308-20:head [setxattr _header (57) in=7b] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.378334+0000", 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 12.725594535000001, 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81367699000000004, 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.193 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 745 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378334+0000", 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378334+0000", 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.194 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378335+0000", 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.257e-06 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378343+0000", 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5789999999999998e-06 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378358+0000", 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4987000000000001e-05 2024-04-26T04:33:15.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378369+0000", 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1353e-05 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.582538+0000", 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20416878699999999 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.582700+0000", 2024-04-26T04:33:15.196 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016191 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.583551+0000", 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00085183999999999998 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.684372+0000", 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100820914 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.191891+0000", 2024-04-26T04:33:15.197 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.507518468 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.192011+0000", 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000119895 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:746 3.ds0 3:bd5491ce:::smithi17829308-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.378386+0000", 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 12.725542755999999, 2024-04-26T04:33:15.198 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2193996330000001, 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 746 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378386+0000", 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.199 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378386+0000", 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378386+0000", 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.1699999999999997e-07 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378390+0000", 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.4400000000000001e-06 2024-04-26T04:33:15.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378403+0000", 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3686000000000001e-05 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378414+0000", 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0976e-05 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.192805+0000", 2024-04-26T04:33:15.201 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81439069900000005 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.193205+0000", 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00040003400000000001 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.597785+0000", 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40457998099999998 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.202 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 67/30 ECSubWriteReply(tid=64, last_complete=67'42, committed=1, applied=1))", 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.478796+0000", 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 12.625132367999999, 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30610332899999998, 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.478796+0000", 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.203 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.205 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.205 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.205 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.478796+0000", 2024-04-26T04:33:15.205 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.205 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.205 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.205 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.205 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.478797+0000", 2024-04-26T04:33:15.205 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.573e-06 2024-04-26T04:33:15.205 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.205 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.478808+0000", 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.096e-05 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.478828+0000", 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9964000000000001e-05 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.578921+0000", 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10009237 2024-04-26T04:33:15.206 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.784875+0000", 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.205954516 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.784899+0000", 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3946000000000001e-05 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.207 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpRead(3.ds0 67/30 ECSubRead(tid=70, to_read={3:bd5491ce:::smithi17829308-20:head=0,4096,0}, subchunks={3:bd5491ce:::smithi17829308-20:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:03.193748+0000", 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 11.910179835999999, 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30233241900000002, 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.193748+0000", 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.208 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.193748+0000", 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.193748+0000", 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.193748+0000", 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.209 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.193790+0000", 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.1279000000000002e-05 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.293934+0000", 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100144097 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.495906+0000", 2024-04-26T04:33:15.210 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201972179 2024-04-26T04:33:15.211 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.211 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.211 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:15.211 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.496081+0000", 2024-04-26T04:33:15.211 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000174864 2024-04-26T04:33:15.211 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.211 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.211 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.211 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.211 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.211 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:15.212 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:33:15.213 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:33:15.213 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:33:15.214 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:33:15.214 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T04:33:15.214 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:15.209+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:15.230 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:33:15.231 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:33:15.251 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:33:15.252 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:33:15.270 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:33:15.270 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:33:15.313 INFO:tasks.rados.rados.0.smithi178.stdout:357: done (4 left) 2024-04-26T04:33:15.314 INFO:tasks.rados.rados.0.smithi178.stdout:358: read oid 39 snap -1 2024-04-26T04:33:15.314 INFO:tasks.rados.rados.0.smithi178.stdout:358: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:33:15.314 INFO:tasks.rados.rados.0.smithi178.stdout:359: delete oid 48 current snap is 23 2024-04-26T04:33:15.314 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 16 v 0 (ObjNum 104 snap 13 seq_num 104) dirty exists 2024-04-26T04:33:15.314 INFO:tasks.rados.rados.0.smithi178.stdout:355: finishing rollback tid 1 to smithi17829308-7 2024-04-26T04:33:15.314 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 7 v 54 (ObjNum 289 snap 0 seq_num 0) dirty dne 2024-04-26T04:33:15.314 INFO:tasks.rados.rados.0.smithi178.stdout:353: finishing write tid 1 to smithi17829308-5 2024-04-26T04:33:15.314 INFO:tasks.rados.rados.0.smithi178.stdout:353: finishing write tid 2 to smithi17829308-5 2024-04-26T04:33:15.314 INFO:tasks.rados.rados.0.smithi178.stdout:351: expect (ObjNum 87 snap 12 seq_num 87) 2024-04-26T04:33:15.315 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:15.308+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:15.347 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:33:15.347 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:33:15.390 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.1s1 80/30 ECSubWrite(tid=348, reqid=osd.4.0:348, at_version=80'88, trim_to=0'0, roll_forward_to=69'86))", 2024-04-26T04:33:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:15.322392+0000", 2024-04-26T04:33:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.067728015000000003, 2024-04-26T04:33:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.067773761000000002, 2024-04-26T04:33:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.390 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "no flag points reached", 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:15.322392+0000", 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:15.322392+0000", 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.391 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:15.322392+0000", 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0199999999999998e-07 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:15.322421+0000", 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8382e-05 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:15.322437+0000", 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6237999999999999e-05 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.392 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 79/30 ECSubWriteReply(tid=147, last_complete=80'112, committed=1, applied=1))", 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:15.328772+0000", 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.061347783000000003, 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.061473287000000001, 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "no flag points reached", 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:15.328772+0000", 2024-04-26T04:33:15.393 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:15.328772+0000", 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:15.328773+0000", 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.6099999999999999e-07 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:15.394 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:15.328781+0000", 2024-04-26T04:33:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.3909999999999998e-06 2024-04-26T04:33:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:15.328797+0000", 2024-04-26T04:33:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5719e-05 2024-04-26T04:33:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-04-26T04:33:15.395 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 2 2024-04-26T04:33:15.395 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:15.406 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T04:33:15.409 INFO:tasks.rados.rados.0.smithi178.stdout:353: finishing write tid 3 to smithi17829308-5 2024-04-26T04:33:15.409 INFO:tasks.rados.rados.0.smithi178.stdout:353: finishing write tid 4 to smithi17829308-5 2024-04-26T04:33:15.414 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:15.412+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:15.448 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:33:15.448 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:33:15.479 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:33:15.480 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:33:15.508 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:33:15.508 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:33:15.515 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:15.516+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:15.528 INFO:tasks.rados.rados.0.smithi178.stdout:353: finishing write tid 5 to smithi17829308-5 2024-04-26T04:33:15.533 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:15.533 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:15.534 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:33:15.534 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:33:15.534 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:15.539 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:33:15.539 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:33:15.549 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T04:33:15.560 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:33:15.561 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:33:15.583 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:33:15.584 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:33:15.617 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:15.616+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:15.646 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:33:15.646 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:33:15.671 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:15.672 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:15.672 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:33:15.673 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:33:15.685 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T04:33:15.701 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:33:15.701 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:33:15.718 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:15.713+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:15.722 INFO:tasks.rados.rados.0.smithi178.stdout:351: done (5 left) 2024-04-26T04:33:15.723 INFO:tasks.rados.rados.0.smithi178.stdout:355: done (4 left) 2024-04-26T04:33:15.723 INFO:tasks.rados.rados.0.smithi178.stdout:356: done (3 left) 2024-04-26T04:33:15.723 INFO:tasks.rados.rados.0.smithi178.stdout:359: done (2 left) 2024-04-26T04:33:15.723 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 25 current snap is 23 2024-04-26T04:33:15.723 INFO:tasks.rados.rados.0.smithi178.stdout:360: seq_num 141 ranges {0=425984,425984=573440,999424=425984,1425408=524288,1949696=425984} 2024-04-26T04:33:15.726 INFO:tasks.rados.rados.0.smithi178.stdout:360: writing smithi17829308-25 from 0 to 425984 tid 1 2024-04-26T04:33:15.731 INFO:tasks.rados.rados.0.smithi178.stdout:360: writing smithi17829308-25 from 425984 to 999424 tid 2 2024-04-26T04:33:15.735 INFO:tasks.rados.rados.0.smithi178.stdout:360: writing smithi17829308-25 from 999424 to 1425408 tid 3 2024-04-26T04:33:15.739 INFO:tasks.rados.rados.0.smithi178.stdout:360: writing smithi17829308-25 from 1425408 to 1949696 tid 4 2024-04-26T04:33:15.743 INFO:tasks.rados.rados.0.smithi178.stdout:360: writing smithi17829308-25 from 1949696 to 2375680 tid 5 2024-04-26T04:33:15.743 INFO:tasks.rados.rados.0.smithi178.stdout:361: snap_remove snap 22 2024-04-26T04:33:15.751 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:33:15.752 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:33:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:15.813 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:15.819 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:15.816+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:15.828 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T04:33:15.919 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:15.918+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:15.993 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:15.994 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:16.009 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T04:33:16.021 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:16.020+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:16.122 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:16.120+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:16.181 INFO:tasks.rados.rados.0.smithi178.stdout:361: done (3 left) 2024-04-26T04:33:16.181 INFO:tasks.rados.rados.0.smithi178.stdout:362: snap_remove snap 14 2024-04-26T04:33:16.182 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:16.182 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:16.198 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T04:33:16.223 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:16.224+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:16.324 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-26T04:33:16.324 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:16.320+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:16.345 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:16.345 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:16.360 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T04:33:16.425 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:16.422+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:16.526 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:16.521+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:16.552 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:16.552 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:16.567 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T04:33:16.627 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:16.628+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:16.728 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:16.726+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:16.733 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:16.734 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:16.748 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T04:33:16.829 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:16.825+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:16.875 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:16.875 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:16.891 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T04:33:16.933 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:16.929+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:17.031 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:17.028+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:17.063 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:17.063 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:17.077 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T04:33:17.132 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:17.132+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:17.174 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-04-26T04:33:17.181 INFO:tasks.rados.rados.0.smithi178.stdout:362: done (3 left) 2024-04-26T04:33:17.181 INFO:tasks.rados.rados.0.smithi178.stdout:363: snap_remove snap 18 2024-04-26T04:33:17.211 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:17.212 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:17.226 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T04:33:17.233 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:17.232+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:17.334 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:17.332+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:17.405 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:17.405 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:17.420 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T04:33:17.435 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:17.436+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:17.536 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:17.529+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:17.543 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:17.544 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:17.558 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T04:33:17.637 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:17.636+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:17.731 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:17.732 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:17.738 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:17.734+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:17.746 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T04:33:17.774 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T04:33:17.839 INFO:tasks.ceph.osd.9.smithi113.stderr:2024-04-26T04:33:17.836+0000 7ff1e53c8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25074) UID: 0 2024-04-26T04:33:17.869 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:17.869 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:17.884 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T04:33:17.939 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:17.936+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:17.942 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:33:17.942 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:33:17.963 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:33:17.963 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:33:17.985 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:33:17.985 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:33:18.004 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:18.005 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:18.006 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:33:18.006 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:33:18.019 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T04:33:18.069 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:18.065+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:18.113 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:33:18.113 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:33:18.134 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:33:18.134 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:33:18.137 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:18.137 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:18.141 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:18.140+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:18.152 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T04:33:18.160 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:33:18.161 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:33:18.182 INFO:tasks.rados.rados.0.smithi178.stdout:363: done (3 left) 2024-04-26T04:33:18.183 INFO:tasks.rados.rados.0.smithi178.stdout:364: snap_remove snap 21 2024-04-26T04:33:18.188 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:33:18.189 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:33:18.209 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:33:18.209 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:33:18.227 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:33:18.228 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:33:18.242 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:18.240+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:18.248 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:33:18.249 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:33:18.286 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:18.286 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:33:18.286 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 0 2024-04-26T04:33:18.286 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:18.300 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T04:33:18.329 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:33:18.329 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:33:18.345 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:18.344+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:18.353 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:33:18.354 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:33:18.374 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:33:18.375 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:33:18.445 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:18.440+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:18.466 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:18.467 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:33:18.467 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:33:18.467 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:33:18.467 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:18.481 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T04:33:18.512 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:33:18.512 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:33:18.546 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:18.548+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:18.615 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [2] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T04:33:18.615 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T04:33:18.615 INFO:tasks.thrashosds.thrasher:Removing osd 10, in_osds are: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] 2024-04-26T04:33:18.615 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 10 2024-04-26T04:33:18.649 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:18.649 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:33:18.649 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:33:18.649 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:33:18.649 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.649 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 45/30 ECSubWrite(tid=31, reqid=client.4670.0:374, at_version=45'20, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:33:18.649 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.532047+0000", 2024-04-26T04:33:18.650 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 39.114542448999998, 2024-04-26T04:33:18.650 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17995593700000001, 2024-04-26T04:33:18.650 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.650 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.650 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.650 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.650 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.650 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.650 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532047+0000", 2024-04-26T04:33:18.650 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.650 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.650 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.650 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532047+0000", 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532048+0000", 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5960000000000001e-06 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532346+0000", 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00029776800000000001 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.651 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532375+0000", 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8915e-05 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.532395+0000", 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9663000000000001e-05 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608323+0000", 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.075928566000000003 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.652 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608336+0000", 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3377000000000001e-05 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.711932+0000", 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10359542099999999 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.712003+0000", 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0630999999999995e-05 2024-04-26T04:33:18.653 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 45/30 ECSubWrite(tid=32, reqid=client.4670.0:375, at_version=45'21, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.533950+0000", 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 39.112638758999999, 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17809028800000001, 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.654 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.533950+0000", 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.533950+0000", 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.533951+0000", 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1200000000000002e-07 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.655 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.656 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.656 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.534075+0000", 2024-04-26T04:33:18.656 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012397999999999999 2024-04-26T04:33:18.656 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.656 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.656 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.656 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.534089+0000", 2024-04-26T04:33:18.656 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4097000000000001e-05 2024-04-26T04:33:18.656 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.656 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.657 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.657 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.534097+0000", 2024-04-26T04:33:18.657 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.6420000000000007e-06 2024-04-26T04:33:18.657 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.658 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.658 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.658 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.709588+0000", 2024-04-26T04:33:18.658 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17549146800000001 2024-04-26T04:33:18.658 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.658 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.658 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.658 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.709604+0000", 2024-04-26T04:33:18.658 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5400000000000002e-05 2024-04-26T04:33:18.658 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.658 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.658 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.658 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.712011+0000", 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0024075469999999999 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.712041+0000", 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9342000000000001e-05 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 45/30 ECSubWrite(tid=33, reqid=client.4670.0:376, at_version=45'22, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.535418+0000", 2024-04-26T04:33:18.659 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 39.111171108999997, 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.177833458, 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535418+0000", 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535418+0000", 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.660 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535419+0000", 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6400000000000001e-07 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535502+0000", 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.365e-05 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535517+0000", 2024-04-26T04:33:18.661 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4489e-05 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.535525+0000", 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4940000000000003e-06 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.710429+0000", 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17490356100000001 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.710454+0000", 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5038999999999999e-05 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.713205+0000", 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00275055 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.713251+0000", 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.6811000000000003e-05 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s3 45/30 ECSubWrite(tid=34, reqid=client.4670.0:377, at_version=45'23, trim_to=0'0, roll_forward_to=44'18))", 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.536571+0000", 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 39.110017554000002, 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17941110099999999, 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536571+0000", 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536571+0000", 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536572+0000", 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.0999999999999998e-07 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536589+0000", 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7110000000000001e-05 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536601+0000", 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1368e-05 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.536608+0000", 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2490000000000004e-06 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.711524+0000", 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17491638100000001 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.711553+0000", 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9142e-05 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.715942+0000", 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00438895 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.667 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.715983+0000", 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.0290999999999998e-05 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 47/30 ECSubWrite(tid=55, reqid=osd.5.0:55, at_version=47'43, trim_to=0'0, roll_forward_to=44'41))", 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:41.211282+0000", 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 37.435306904999997, 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.25277038499999999, 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.668 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211282+0000", 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211282+0000", 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211283+0000", 2024-04-26T04:33:18.669 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6400000000000002e-07 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211337+0000", 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.4304e-05 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211361+0000", 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3983000000000001e-05 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.211395+0000", 2024-04-26T04:33:18.670 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4255000000000003e-05 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.363290+0000", 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.15189467800000001 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.363307+0000", 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6863000000000001e-05 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.463984+0000", 2024-04-26T04:33:18.671 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100677144 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.464053+0000", 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.8493999999999998e-05 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "pg_update_log_missing(3.8s3 epoch 50/30 rep_tid 66 entries 50'54 (0'0) error 3:1d1feae2:::smithi17829308-1:head by client.4670.0:422 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 50'51)", 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:44.585396+0000", 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 34.061193134, 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20079079399999999, 2024-04-26T04:33:18.672 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585396+0000", 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585396+0000", 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.673 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585398+0000", 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7999999999999999e-06 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585418+0000", 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0358999999999999e-05 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.585468+0000", 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.9444999999999998e-05 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.674 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.685596+0000", 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100128605 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.685857+0000", 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026076799999999998 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:44.786187+0000", 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100329817 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.675 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.fs1 52/30 ECSubWrite(tid=57, reqid=client.4670.0:431, at_version=52'36, trim_to=0'0, roll_forward_to=48'33))", 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:46.434158+0000", 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 32.212430675, 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20233947599999999, 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.676 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.434158+0000", 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.434158+0000", 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.434159+0000", 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7300000000000005e-07 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.677 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.434425+0000", 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026552800000000001 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.434448+0000", 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3425e-05 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.534531+0000", 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10008265700000001 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.678 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.679 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.679 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.534635+0000", 2024-04-26T04:33:18.679 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000104326 2024-04-26T04:33:18.679 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.679 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.679 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.679 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.534644+0000", 2024-04-26T04:33:18.679 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.551e-06 2024-04-26T04:33:18.679 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.679 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.679 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.679 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.636402+0000", 2024-04-26T04:33:18.679 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10175832899999999 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:46.636498+0000", 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.5886999999999999e-05 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.fs1 60/30 ECSubWrite(tid=89, reqid=client.4670.0:607, at_version=60'56, trim_to=0'0, roll_forward_to=59'54))", 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.876722+0000", 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.769867108, 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20298210799999999, 2024-04-26T04:33:18.680 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.876722+0000", 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.876722+0000", 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.681 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.876723+0000", 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0599999999999999e-07 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.876913+0000", 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000190669 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.876928+0000", 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.503e-05 2024-04-26T04:33:18.682 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.685 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.685 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.685 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.876938+0000", 2024-04-26T04:33:18.685 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3489999999999998e-06 2024-04-26T04:33:18.685 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.685 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.685 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.685 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.077326+0000", 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20038803099999999 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.077352+0000", 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6533000000000002e-05 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.079616+0000", 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0022639090000000001 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.686 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.079704+0000", 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7781000000000005e-05 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.as2 60/30 ECSubRead(tid=47, to_read={3:57b6ca82:::smithi17829308-14:head=0,57344,0}, subchunks={3:57b6ca82:::smithi17829308-14:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:56.212241+0000", 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.434347580000001, 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17507404200000001, 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.687 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.212241+0000", 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.212241+0000", 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.688 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.212242+0000", 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9999999999999998e-07 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.212254+0000", 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1686e-05 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.212291+0000", 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.7778999999999999e-05 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.689 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.212311+0000", 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9726000000000001e-05 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.387083+0000", 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.17477169000000001 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.387316+0000", 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000232761 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.690 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 61/30 ECSubWrite(tid=98, reqid=client.4670.0:624, at_version=61'76, trim_to=0'0, roll_forward_to=60'71))", 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:56.636689+0000", 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.009900402, 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.203428576, 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636689+0000", 2024-04-26T04:33:18.691 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636689+0000", 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636691+0000", 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8840000000000001e-06 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.692 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636916+0000", 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0002252 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636947+0000", 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0973000000000003e-05 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636967+0000", 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0041999999999999e-05 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.693 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.737233+0000", 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10026647800000001 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.737252+0000", 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8508999999999998e-05 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.840059+0000", 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102807022 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.694 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.840117+0000", 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.8468000000000003e-05 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 61/30 ECSubWrite(tid=99, reqid=client.4670.0:625, at_version=61'77, trim_to=0'0, roll_forward_to=60'73))", 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:56.637006+0000", 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.009583154000001, 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20363767699999999, 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.695 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637006+0000", 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637006+0000", 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.696 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637007+0000", 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.063e-06 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637211+0000", 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020410200000000001 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637231+0000", 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9508999999999999e-05 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.697 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637243+0000", 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2490000000000001e-05 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.838505+0000", 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20126216099999999 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.838520+0000", 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.486e-05 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.698 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.840583+0000", 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0020633079999999998 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.840644+0000", 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.0183999999999998e-05 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.699 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 61/30 ECSubWrite(tid=100, reqid=client.4670.0:626, at_version=61'78, trim_to=0'0, roll_forward_to=60'73))", 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:56.637270+0000", 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.009318898, 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.203845515, 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637270+0000", 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.700 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637270+0000", 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637271+0000", 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7700000000000004e-07 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637358+0000", 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6990999999999994e-05 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.701 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637375+0000", 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7125999999999998e-05 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637385+0000", 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.043e-05 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.839422+0000", 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20203700699999999 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.702 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.839438+0000", 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5993000000000001e-05 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.841059+0000", 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016203999999999999 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.841116+0000", 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.6790999999999999e-05 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.703 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 61/30 ECSubWrite(tid=101, reqid=client.4670.0:627, at_version=61'79, trim_to=0'0, roll_forward_to=60'73))", 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:56.637402+0000", 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.009187197999999, 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.204692506, 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.704 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637402+0000", 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637402+0000", 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637402+0000", 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.37e-07 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.705 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637419+0000", 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6555999999999999e-05 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637433+0000", 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.403e-05 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.637442+0000", 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.4520000000000003e-06 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.706 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.840213+0000", 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20277055499999999 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.840229+0000", 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5752000000000001e-05 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.842033+0000", 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001804625 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.707 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.842094+0000", 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.0999000000000003e-05 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.as2 66/30 ECSubWrite(tid=52, reqid=client.4670.0:702, at_version=66'39, trim_to=0'0, roll_forward_to=64'37))", 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:02.109720+0000", 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 16.536868999999999, 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20405148200000001, 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.708 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.109720+0000", 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.109720+0000", 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.109723+0000", 2024-04-26T04:33:18.709 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8930000000000001e-06 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.111124+0000", 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001400966 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.111172+0000", 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.8463e-05 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.710 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.211276+0000", 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100104046 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.311564+0000", 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10028720000000001 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.311578+0000", 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4283e-05 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.711 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.313678+0000", 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.002100509 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:02.313772+0000", 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3121999999999994e-05 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs2 68/30 ECSubWrite(tid=273, reqid=osd.4.0:273, at_version=68'45, trim_to=0'0, roll_forward_to=66'43))", 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:03.437366+0000", 2024-04-26T04:33:18.712 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 15.209222910999999, 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20151760899999999, 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.437366+0000", 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.437366+0000", 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.713 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.437367+0000", 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.0999999999999999e-07 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.437406+0000", 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9452000000000001e-05 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.437426+0000", 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0273000000000001e-05 2024-04-26T04:33:18.714 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.437442+0000", 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5758999999999999e-05 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.537703+0000", 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100260827 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.537732+0000", 2024-04-26T04:33:18.715 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8666e-05 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.638854+0000", 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101122817 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.638884+0000", 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9405000000000001e-05 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.716 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs2 68/30 ECSubWrite(tid=274, reqid=osd.4.0:274, at_version=68'47, trim_to=0'0, roll_forward_to=66'43))", 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:03.439174+0000", 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 15.207415297000001, 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20031373399999999, 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.439174+0000", 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.717 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.439174+0000", 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.439175+0000", 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.584e-06 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.439212+0000", 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.6338999999999999e-05 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.718 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.439241+0000", 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9009e-05 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.439256+0000", 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.518e-05 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.638575+0000", 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19931944200000001 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.719 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.638583+0000", 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7770000000000002e-06 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.639444+0000", 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00086127100000000004 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.639487+0000", 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.3132e-05 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.720 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 68/30 ECSubWrite(tid=110, reqid=osd.5.0:110, at_version=68'85, trim_to=0'0, roll_forward_to=68'83))", 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:03.837614+0000", 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 14.808974796999999, 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20175926399999999, 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.837614+0000", 2024-04-26T04:33:18.721 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.837614+0000", 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.837615+0000", 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.6599999999999996e-07 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.722 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.837643+0000", 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8150999999999999e-05 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.837658+0000", 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5064e-05 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.837667+0000", 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4800000000000001e-06 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.723 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.937849+0000", 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10018202900000001 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.937865+0000", 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6855999999999999e-05 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:04.039298+0000", 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10143284700000001 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.724 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:04.039374+0000", 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5171000000000002e-05 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.8s3 68/30 ECSubWrite(tid=111, reqid=osd.5.0:111, at_version=68'87, trim_to=0'0, roll_forward_to=68'83))", 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:03.838468+0000", 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 14.808120575, 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201938285, 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.725 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.838468+0000", 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.838468+0000", 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.838469+0000", 2024-04-26T04:33:18.726 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3e-07 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.838503+0000", 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.3767999999999999e-05 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.838520+0000", 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6685e-05 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.838529+0000", 2024-04-26T04:33:18.727 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.4360000000000008e-06 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:04.038696+0000", 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20016673600000001 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:04.038711+0000", 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5753e-05 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.728 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:04.040304+0000", 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001592169 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:04.040407+0000", 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000103108 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs2 71/30 ECSubWrite(tid=299, reqid=client.4670.0:803, at_version=71'55, trim_to=0'0, roll_forward_to=71'54))", 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:07.215303+0000", 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 11.431285886, 2024-04-26T04:33:18.729 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20347427900000001, 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.215303+0000", 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.215303+0000", 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.730 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.215305+0000", 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.824e-06 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.215651+0000", 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00034581999999999999 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.215685+0000", 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4391999999999999e-05 2024-04-26T04:33:18.731 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.215705+0000", 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0095999999999999e-05 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.316039+0000", 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100333461 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.316055+0000", 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5795999999999999e-05 2024-04-26T04:33:18.732 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.418721+0000", 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102666486 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.418777+0000", 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.6403999999999997e-05 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.733 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs2 79/30 ECSubWrite(tid=342, reqid=osd.4.0:342, at_version=79'75, trim_to=0'0, roll_forward_to=78'71))", 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:14.351472+0000", 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.295117104, 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.302179476, 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.351472+0000", 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.734 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.351472+0000", 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.351474+0000", 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7430000000000001e-06 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.351551+0000", 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.6893999999999999e-05 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.735 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.351582+0000", 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1758999999999999e-05 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.451679+0000", 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100096276 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.652003+0000", 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20032439299999999 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.736 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.652019+0000", 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6322999999999999e-05 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.653587+0000", 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0015679839999999999 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.653651+0000", 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.4103999999999995e-05 2024-04-26T04:33:18.737 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.738 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.738 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.738 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.738 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.738 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:18.739 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:33:18.657+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:33:18.739 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T04:33:18.747 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:33:18.748 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:33:18.748 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:18.741+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:18.849 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:33:18.845+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:33:18.873 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:33:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.873 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 83/30 ECSubWriteReply(tid=146, last_complete=83'126, committed=1, applied=1))", 2024-04-26T04:33:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:18.787259+0000", 2024-04-26T04:33:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.086656638999999994, 2024-04-26T04:33:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.086712213999999996, 2024-04-26T04:33:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "no flag points reached", 2024-04-26T04:33:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.787259+0000", 2024-04-26T04:33:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.874 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.787259+0000", 2024-04-26T04:33:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.787261+0000", 2024-04-26T04:33:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7209999999999999e-06 2024-04-26T04:33:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.875 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.787270+0000", 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.5349999999999993e-06 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.787288+0000", 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7651000000000001e-05 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 1 2024-04-26T04:33:18.876 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:18.888 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T04:33:18.947 INFO:tasks.ceph.mon.a.smithi012.stderr:2024-04-26T04:33:18.941+0000 7f05ec032640 -1 mon.a@0(leader).osd e83 definitely_dead 0 2024-04-26T04:33:18.949 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:18.946+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:19.004 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:19.004 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:33:19.005 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:33:19.005 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:33:19.005 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:19.020 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T04:33:19.051 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:19.046+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:19.145 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:33:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:33:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:33:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:247 3.8s0 3:1a4ccad9:::smithi17829308-40:head [append 0~573440] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:19.145 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.788189+0000", 2024-04-26T04:33:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 49.353983816000003, 2024-04-26T04:33:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.58923427699999997, 2024-04-26T04:33:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 247 2024-04-26T04:33:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.146 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.147 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.147 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.147 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.788189+0000", 2024-04-26T04:33:19.147 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.147 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.147 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.147 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.147 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.788189+0000", 2024-04-26T04:33:19.147 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.147 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.147 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.147 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.147 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.788190+0000", 2024-04-26T04:33:19.148 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0270000000000001e-06 2024-04-26T04:33:19.148 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.148 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.148 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.148 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.789159+0000", 2024-04-26T04:33:19.148 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00096895399999999995 2024-04-26T04:33:19.148 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.148 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.148 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.148 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.789190+0000", 2024-04-26T04:33:19.148 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0839000000000001e-05 2024-04-26T04:33:19.148 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.148 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.149 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.149 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.789205+0000", 2024-04-26T04:33:19.149 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5139e-05 2024-04-26T04:33:19.149 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.149 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.149 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.149 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072824+0000", 2024-04-26T04:33:19.150 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.283618659 2024-04-26T04:33:19.150 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.150 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.150 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.150 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.072888+0000", 2024-04-26T04:33:19.150 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.4080999999999999e-05 2024-04-26T04:33:19.150 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.151 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.151 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.151 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.073554+0000", 2024-04-26T04:33:19.151 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00066656699999999996 2024-04-26T04:33:19.151 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.151 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.151 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.151 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.077266+0000", 2024-04-26T04:33:19.151 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0037117320000000001 2024-04-26T04:33:19.151 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.151 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.152 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.152 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.377359+0000", 2024-04-26T04:33:19.152 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30009325399999998 2024-04-26T04:33:19.152 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.152 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.152 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.152 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.377423+0000", 2024-04-26T04:33:19.152 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.4024999999999998e-05 2024-04-26T04:33:19.152 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.152 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.153 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.153 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.153 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.153 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:248 3.8s0 3:1a4ccad9:::smithi17829308-40:head [append 0~376832] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:19.153 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.793207+0000", 2024-04-26T04:33:19.153 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 49.348965800000002, 2024-04-26T04:33:19.153 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.68466394600000002, 2024-04-26T04:33:19.153 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.153 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.153 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.154 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.154 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.154 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.154 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 248 2024-04-26T04:33:19.154 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.154 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.154 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.154 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.154 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793207+0000", 2024-04-26T04:33:19.154 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.154 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.154 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.155 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.155 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793207+0000", 2024-04-26T04:33:19.155 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.155 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.155 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.155 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.155 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793209+0000", 2024-04-26T04:33:19.155 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7269999999999999e-06 2024-04-26T04:33:19.155 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.155 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.156 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.156 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793709+0000", 2024-04-26T04:33:19.156 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000500247 2024-04-26T04:33:19.156 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.156 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.156 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.156 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793730+0000", 2024-04-26T04:33:19.156 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0931999999999999e-05 2024-04-26T04:33:19.156 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.156 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.156 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793741+0000", 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1032999999999999e-05 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.074358+0000", 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.28061672700000001 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.074434+0000", 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.6691000000000001e-05 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.157 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.158 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.075231+0000", 2024-04-26T04:33:19.158 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00079667900000000001 2024-04-26T04:33:19.158 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.158 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.158 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.158 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.077267+0000", 2024-04-26T04:33:19.158 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0020362380000000001 2024-04-26T04:33:19.158 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.158 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.158 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.158 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.477751+0000", 2024-04-26T04:33:19.158 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.400483435 2024-04-26T04:33:19.158 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.477871+0000", 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012023699999999999 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:249 3.8s0 3:1a4ccad9:::smithi17829308-40:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.793759+0000", 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 49.348413762, 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.68433669600000002, 2024-04-26T04:33:19.159 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.160 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.160 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.160 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.160 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.160 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.160 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 249 2024-04-26T04:33:19.160 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.160 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.160 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.160 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.161 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793759+0000", 2024-04-26T04:33:19.161 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.161 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.161 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.161 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.161 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793759+0000", 2024-04-26T04:33:19.161 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.161 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.162 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.162 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.162 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793760+0000", 2024-04-26T04:33:19.162 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.043e-06 2024-04-26T04:33:19.162 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.162 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.162 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.162 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793765+0000", 2024-04-26T04:33:19.162 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.51e-06 2024-04-26T04:33:19.162 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.162 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.162 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.163 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.793775+0000", 2024-04-26T04:33:19.163 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0586e-05 2024-04-26T04:33:19.163 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.163 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.163 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.163 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893886+0000", 2024-04-26T04:33:19.163 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100111197 2024-04-26T04:33:19.163 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.163 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.163 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.163 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.075921+0000", 2024-04-26T04:33:19.163 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.18203503800000001 2024-04-26T04:33:19.163 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.164 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.164 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.164 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.076055+0000", 2024-04-26T04:33:19.164 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013402999999999999 2024-04-26T04:33:19.164 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.164 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.164 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.164 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.076679+0000", 2024-04-26T04:33:19.164 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00062310000000000002 2024-04-26T04:33:19.164 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.164 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.164 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.165 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.178214+0000", 2024-04-26T04:33:19.165 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101535002 2024-04-26T04:33:19.165 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.165 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.165 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.165 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478030+0000", 2024-04-26T04:33:19.165 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.29981632200000002 2024-04-26T04:33:19.165 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.165 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.165 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.165 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478096+0000", 2024-04-26T04:33:19.165 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.5868e-05 2024-04-26T04:33:19.165 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.166 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.166 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.166 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.166 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.166 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:250 3.8s0 3:1a4ccad9:::smithi17829308-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:19.166 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.893915+0000", 2024-04-26T04:33:19.167 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 49.248258307999997, 2024-04-26T04:33:19.167 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.66902760400000005, 2024-04-26T04:33:19.167 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.167 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.167 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:19.167 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.167 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.167 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.167 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 250 2024-04-26T04:33:19.167 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.167 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.167 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893915+0000", 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893915+0000", 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893917+0000", 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3029999999999998e-06 2024-04-26T04:33:19.168 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.169 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.169 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.169 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893925+0000", 2024-04-26T04:33:19.169 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4370000000000007e-06 2024-04-26T04:33:19.169 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.169 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.169 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.169 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893965+0000", 2024-04-26T04:33:19.169 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9941000000000003e-05 2024-04-26T04:33:19.169 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.169 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.169 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.170 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.893976+0000", 2024-04-26T04:33:19.170 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1049999999999999e-05 2024-04-26T04:33:19.170 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.170 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.170 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.170 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.177679+0000", 2024-04-26T04:33:19.170 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.28370258100000001 2024-04-26T04:33:19.170 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.170 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.171 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:19.171 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.177965+0000", 2024-04-26T04:33:19.171 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000286309 2024-04-26T04:33:19.171 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.171 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.171 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.171 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478480+0000", 2024-04-26T04:33:19.171 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30051439600000002 2024-04-26T04:33:19.171 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.171 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.171 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.171 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.478666+0000", 2024-04-26T04:33:19.172 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018635599999999999 2024-04-26T04:33:19.172 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.172 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.172 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.172 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.562942+0000", 2024-04-26T04:33:19.172 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.084276230999999993 2024-04-26T04:33:19.172 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.172 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.172 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.172 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.172 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.172 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:472 3.8s0 3:1d1feae2:::smithi17829308-1:head [append 0~688128] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:19.172 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:50.191944+0000", 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 28.950228446000001, 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.399307472, 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 472 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.191944+0000", 2024-04-26T04:33:19.173 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.174 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.174 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.174 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.174 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.191944+0000", 2024-04-26T04:33:19.174 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.174 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.174 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.174 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.175 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.191947+0000", 2024-04-26T04:33:19.175 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.01e-06 2024-04-26T04:33:19.175 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.175 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.175 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.175 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.193010+0000", 2024-04-26T04:33:19.175 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0010629599999999999 2024-04-26T04:33:19.176 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.176 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.176 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.176 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.193042+0000", 2024-04-26T04:33:19.176 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1334999999999997e-05 2024-04-26T04:33:19.176 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.177 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.177 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.177 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.193061+0000", 2024-04-26T04:33:19.177 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9432999999999998e-05 2024-04-26T04:33:19.177 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.177 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.178 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.178 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.285076+0000", 2024-04-26T04:33:19.178 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.092015206000000002 2024-04-26T04:33:19.178 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.178 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.178 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.179 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.285194+0000", 2024-04-26T04:33:19.179 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000117422 2024-04-26T04:33:19.179 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.179 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.179 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.179 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.286738+0000", 2024-04-26T04:33:19.179 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001544557 2024-04-26T04:33:19.180 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.180 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.180 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.180 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.396638+0000", 2024-04-26T04:33:19.180 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.109899996 2024-04-26T04:33:19.180 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.180 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.181 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.181 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.591133+0000", 2024-04-26T04:33:19.181 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19449470299999999 2024-04-26T04:33:19.181 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.181 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.181 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.182 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.591252+0000", 2024-04-26T04:33:19.182 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00011885 2024-04-26T04:33:19.182 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.182 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.182 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.182 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.183 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.183 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:473 3.8s0 3:1d1feae2:::smithi17829308-1:head [append 0~540672] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:19.183 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:50.195861+0000", 2024-04-26T04:33:19.183 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 28.946311411, 2024-04-26T04:33:19.183 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.39564706900000002, 2024-04-26T04:33:19.184 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.184 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.184 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.184 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.184 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.184 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.184 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 473 2024-04-26T04:33:19.185 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.185 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.185 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.185 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.185 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.195861+0000", 2024-04-26T04:33:19.185 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.185 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.185 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.185 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.186 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.195861+0000", 2024-04-26T04:33:19.186 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.186 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.186 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.186 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.186 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.195863+0000", 2024-04-26T04:33:19.186 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.629e-06 2024-04-26T04:33:19.186 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.186 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.186 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.186 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.196734+0000", 2024-04-26T04:33:19.186 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00087080299999999996 2024-04-26T04:33:19.186 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.196757+0000", 2024-04-26T04:33:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3309e-05 2024-04-26T04:33:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.196769+0000", 2024-04-26T04:33:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.184e-05 2024-04-26T04:33:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.187 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.288007+0000", 2024-04-26T04:33:19.188 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.091238373999999997 2024-04-26T04:33:19.188 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.188 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.188 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.188 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.288255+0000", 2024-04-26T04:33:19.188 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024784900000000002 2024-04-26T04:33:19.188 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.188 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.188 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.188 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.290014+0000", 2024-04-26T04:33:19.188 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0017592230000000001 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.396649+0000", 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.106634174 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.591413+0000", 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.194764204 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.189 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.591508+0000", 2024-04-26T04:33:19.190 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.5663999999999998e-05 2024-04-26T04:33:19.190 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.190 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.190 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.190 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.190 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.190 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:475 3.8s0 3:1d1feae2:::smithi17829308-1:head [setxattr _header (56) in=7b] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:19.190 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:50.198029+0000", 2024-04-26T04:33:19.190 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 28.944143453999999, 2024-04-26T04:33:19.190 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.39986491299999999, 2024-04-26T04:33:19.190 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.190 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.190 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 475 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198029+0000", 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198029+0000", 2024-04-26T04:33:19.191 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.192 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.192 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.192 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.192 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198031+0000", 2024-04-26T04:33:19.192 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1429999999999999e-06 2024-04-26T04:33:19.192 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.192 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.192 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.192 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198046+0000", 2024-04-26T04:33:19.192 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5123000000000001e-05 2024-04-26T04:33:19.192 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.193 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.193 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.193 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198056+0000", 2024-04-26T04:33:19.193 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0023e-05 2024-04-26T04:33:19.193 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.193 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.193 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.193 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198063+0000", 2024-04-26T04:33:19.195 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2629999999999997e-06 2024-04-26T04:33:19.195 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.195 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.195 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.195 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.294144+0000", 2024-04-26T04:33:19.195 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.096080856000000006 2024-04-26T04:33:19.196 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.196 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.196 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.196 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.294294+0000", 2024-04-26T04:33:19.196 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014994499999999999 2024-04-26T04:33:19.196 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.196 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.196 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.197 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.295203+0000", 2024-04-26T04:33:19.197 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00090927399999999995 2024-04-26T04:33:19.197 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.197 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.197 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.197 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.396653+0000", 2024-04-26T04:33:19.197 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101450493 2024-04-26T04:33:19.197 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.197 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.198 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.198 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.597719+0000", 2024-04-26T04:33:19.198 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201065415 2024-04-26T04:33:19.198 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.198 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.198 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.198 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.597894+0000", 2024-04-26T04:33:19.198 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017537799999999999 2024-04-26T04:33:19.199 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.199 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.199 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.199 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.199 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.199 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:476 3.8s0 3:1d1feae2:::smithi17829308-1:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:19.199 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:50.198078+0000", 2024-04-26T04:33:19.199 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 28.944094498999998, 2024-04-26T04:33:19.200 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70227189199999995, 2024-04-26T04:33:19.200 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.200 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.200 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:19.200 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.200 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.200 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.200 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 476 2024-04-26T04:33:19.200 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.200 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.200 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.200 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.200 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198078+0000", 2024-04-26T04:33:19.201 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.201 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.201 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.201 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.201 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198078+0000", 2024-04-26T04:33:19.201 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.201 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.201 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.201 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.201 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198079+0000", 2024-04-26T04:33:19.201 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.2299999999999999e-07 2024-04-26T04:33:19.201 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.202 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.202 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.202 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198082+0000", 2024-04-26T04:33:19.202 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1700000000000001e-06 2024-04-26T04:33:19.202 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.202 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.202 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.202 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198096+0000", 2024-04-26T04:33:19.202 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3492999999999999e-05 2024-04-26T04:33:19.202 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.202 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.203 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.203 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.198104+0000", 2024-04-26T04:33:19.203 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.6799999999999993e-06 2024-04-26T04:33:19.203 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.203 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.203 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.203 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.395804+0000", 2024-04-26T04:33:19.203 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.197700243 2024-04-26T04:33:19.203 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.203 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.204 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:19.204 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.395966+0000", 2024-04-26T04:33:19.204 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016191799999999999 2024-04-26T04:33:19.204 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.204 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.204 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.204 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.598550+0000", 2024-04-26T04:33:19.204 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.202584176 2024-04-26T04:33:19.204 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.204 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.204 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.204 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.598687+0000", 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000136716 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:50.900350+0000", 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30166357300000002 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:623 3.8s0 3:148e36d6:::smithi17829308-35:head [append 0~622592] snapc e=[e,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.814974+0000", 2024-04-26T04:33:19.205 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 23.327199100000001, 2024-04-26T04:33:19.206 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.724759025, 2024-04-26T04:33:19.206 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.206 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.206 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.206 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.206 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.206 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.206 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 623 2024-04-26T04:33:19.206 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.206 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.206 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.206 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.814974+0000", 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.814974+0000", 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.814975+0000", 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2330000000000001e-06 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.207 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.815521+0000", 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000546052 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.815539+0000", 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7765e-05 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.815549+0000", 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.022e-05 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.208 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.209 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.209 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.321677+0000", 2024-04-26T04:33:19.209 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.50612808300000001 2024-04-26T04:33:19.209 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.209 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.209 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.209 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.321994+0000", 2024-04-26T04:33:19.209 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000316514 2024-04-26T04:33:19.209 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.209 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.209 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.209 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.324954+0000", 2024-04-26T04:33:19.209 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.002960383 2024-04-26T04:33:19.210 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.210 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.210 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.210 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.331369+0000", 2024-04-26T04:33:19.210 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0064146660000000003 2024-04-26T04:33:19.210 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.210 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.211 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.211 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.539554+0000", 2024-04-26T04:33:19.211 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20818551699999999 2024-04-26T04:33:19.211 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.211 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.211 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.211 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.539733+0000", 2024-04-26T04:33:19.211 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017859199999999999 2024-04-26T04:33:19.212 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.212 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.212 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.212 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.212 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.212 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:624 3.8s0 3:148e36d6:::smithi17829308-35:head [append 0~442368] snapc e=[e,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:19.213 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.817427+0000", 2024-04-26T04:33:19.213 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 23.324746226999999, 2024-04-26T04:33:19.213 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1242959029999999, 2024-04-26T04:33:19.213 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.213 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.213 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.213 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.213 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.214 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.214 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 624 2024-04-26T04:33:19.214 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.214 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.214 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.214 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.215 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.817427+0000", 2024-04-26T04:33:19.215 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.215 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.215 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.215 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.215 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.817427+0000", 2024-04-26T04:33:19.216 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.216 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.216 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.216 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.216 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.817428+0000", 2024-04-26T04:33:19.216 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8509999999999999e-06 2024-04-26T04:33:19.216 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.216 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.217 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.217 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.817869+0000", 2024-04-26T04:33:19.217 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00044101299999999997 2024-04-26T04:33:19.217 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.217 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.217 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.217 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.817888+0000", 2024-04-26T04:33:19.217 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8805999999999999e-05 2024-04-26T04:33:19.218 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.218 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.218 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.218 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.817899+0000", 2024-04-26T04:33:19.218 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0237e-05 2024-04-26T04:33:19.218 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.218 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.219 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.219 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.535834+0000", 2024-04-26T04:33:19.219 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.71793524399999997 2024-04-26T04:33:19.219 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.219 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.219 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.219 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.536014+0000", 2024-04-26T04:33:19.220 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018016199999999999 2024-04-26T04:33:19.220 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.220 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.220 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.220 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.537405+0000", 2024-04-26T04:33:19.220 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0013915850000000001 2024-04-26T04:33:19.221 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.221 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.221 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.221 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.544269+0000", 2024-04-26T04:33:19.221 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.006863231 2024-04-26T04:33:19.221 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.221 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.222 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.222 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.941591+0000", 2024-04-26T04:33:19.222 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.39732218800000002 2024-04-26T04:33:19.222 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.222 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.222 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.222 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.941723+0000", 2024-04-26T04:33:19.222 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000131586 2024-04-26T04:33:19.222 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.222 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.222 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.223 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.223 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.223 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:625 3.8s0 3:148e36d6:::smithi17829308-35:head [append 0~524288] snapc e=[e,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:19.223 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.820584+0000", 2024-04-26T04:33:19.223 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 23.321588791, 2024-04-26T04:33:19.223 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.121569483, 2024-04-26T04:33:19.223 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.223 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.223 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.223 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.224 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.224 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.224 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 625 2024-04-26T04:33:19.224 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.224 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.224 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.224 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.224 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.820584+0000", 2024-04-26T04:33:19.224 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.224 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.224 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.224 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.820584+0000", 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.820585+0000", 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4589999999999999e-06 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821101+0000", 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00051561399999999996 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.225 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.226 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.226 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821119+0000", 2024-04-26T04:33:19.226 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8054999999999999e-05 2024-04-26T04:33:19.226 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.226 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.226 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.226 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821130+0000", 2024-04-26T04:33:19.226 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1089e-05 2024-04-26T04:33:19.226 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.226 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.226 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.226 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.539869+0000", 2024-04-26T04:33:19.226 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.71873915499999996 2024-04-26T04:33:19.227 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.227 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.227 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.227 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.540052+0000", 2024-04-26T04:33:19.227 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018266199999999999 2024-04-26T04:33:19.227 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.227 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.227 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.227 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.541436+0000", 2024-04-26T04:33:19.227 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001384363 2024-04-26T04:33:19.227 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.227 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.228 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.228 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.545635+0000", 2024-04-26T04:33:19.228 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0041985219999999997 2024-04-26T04:33:19.228 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.228 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.228 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.229 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.941934+0000", 2024-04-26T04:33:19.229 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.39629892900000002 2024-04-26T04:33:19.229 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.229 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.229 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.229 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.942154+0000", 2024-04-26T04:33:19.229 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021963500000000001 2024-04-26T04:33:19.229 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.229 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.229 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.229 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.230 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.230 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:626 3.8s0 3:148e36d6:::smithi17829308-35:head [append 0~196608] snapc e=[e,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:19.230 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.821693+0000", 2024-04-26T04:33:19.230 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 23.320479811999999, 2024-04-26T04:33:19.230 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1208058139999999, 2024-04-26T04:33:19.230 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.230 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.230 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.230 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.230 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 626 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821693+0000", 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821693+0000", 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.231 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821694+0000", 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0210000000000001e-06 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821910+0000", 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021605499999999999 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821925+0000", 2024-04-26T04:33:19.232 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4931e-05 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821933+0000", 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1890000000000007e-06 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.542462+0000", 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.72052851399999995 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.542611+0000", 2024-04-26T04:33:19.233 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014969199999999999 2024-04-26T04:33:19.234 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.234 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.234 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.234 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.543633+0000", 2024-04-26T04:33:19.234 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001021599 2024-04-26T04:33:19.234 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.235 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.235 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.235 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.545642+0000", 2024-04-26T04:33:19.235 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0020087630000000002 2024-04-26T04:33:19.235 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.235 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.235 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.235 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.942386+0000", 2024-04-26T04:33:19.235 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.396744659 2024-04-26T04:33:19.236 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.236 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.236 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.236 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.942499+0000", 2024-04-26T04:33:19.236 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000112391 2024-04-26T04:33:19.236 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.236 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.236 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.236 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.237 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.237 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:627 3.8s0 3:148e36d6:::smithi17829308-35:head [setxattr _header (57) in=7b] snapc e=[e,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:19.237 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.821959+0000", 2024-04-26T04:33:19.237 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 23.320213588000001, 2024-04-26T04:33:19.237 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1209552119999999, 2024-04-26T04:33:19.237 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.237 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.237 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.237 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.237 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.237 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.238 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 627 2024-04-26T04:33:19.238 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.238 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.238 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.238 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.238 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821959+0000", 2024-04-26T04:33:19.239 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.239 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.239 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.239 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.239 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821959+0000", 2024-04-26T04:33:19.239 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.240 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.240 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.240 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.240 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821960+0000", 2024-04-26T04:33:19.240 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2099999999999996e-07 2024-04-26T04:33:19.240 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.241 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.241 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.241 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821964+0000", 2024-04-26T04:33:19.241 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8759999999999998e-06 2024-04-26T04:33:19.241 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.241 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.242 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.242 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821974+0000", 2024-04-26T04:33:19.242 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.8169999999999994e-06 2024-04-26T04:33:19.242 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.242 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.243 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.243 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821982+0000", 2024-04-26T04:33:19.243 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8550000000000002e-06 2024-04-26T04:33:19.243 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.243 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.243 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.244 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.544392+0000", 2024-04-26T04:33:19.244 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.72241027499999999 2024-04-26T04:33:19.244 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.244 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.244 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.244 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.544530+0000", 2024-04-26T04:33:19.245 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000137832 2024-04-26T04:33:19.245 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.245 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.245 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.245 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.545257+0000", 2024-04-26T04:33:19.245 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000727351 2024-04-26T04:33:19.246 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.246 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.246 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.246 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.547660+0000", 2024-04-26T04:33:19.246 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0024034809999999998 2024-04-26T04:33:19.247 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.247 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.247 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.247 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.942747+0000", 2024-04-26T04:33:19.247 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.39508694900000002 2024-04-26T04:33:19.247 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.247 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.247 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.247 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.942914+0000", 2024-04-26T04:33:19.247 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016705499999999999 2024-04-26T04:33:19.247 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.247 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.247 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:628 3.8s0 3:148e36d6:::smithi17829308-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.821991+0000", 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 23.320182095, 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3245125230000001, 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 628 2024-04-26T04:33:19.248 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821991+0000", 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821991+0000", 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.249 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821991+0000", 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.1900000000000003e-07 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.821994+0000", 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3429999999999998e-06 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.822002+0000", 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6750000000000008e-06 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.250 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922073+0000", 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100070414 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636093+0000", 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.71402036099999999 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.636365+0000", 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00027212599999999998 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.251 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.044197+0000", 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.40783160699999998 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.044341+0000", 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000144463 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.146503+0000", 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10216201499999999 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.252 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.0s4 60/30 ECSubWrite(tid=31, reqid=client.4670.0:605, at_version=60'14, trim_to=0'0, roll_forward_to=59'11))", 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:55.922386+0000", 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 23.219786873, 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.40454453299999998, 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.253 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922386+0000", 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922386+0000", 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922386+0000", 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0899999999999997e-07 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.254 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922407+0000", 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1024999999999999e-05 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922415+0000", 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.96e-06 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:55.922420+0000", 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.5159999999999996e-06 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.255 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.114656+0000", 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.192236623 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.114686+0000", 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0105999999999999e-05 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.326873+0000", 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.212186545 2024-04-26T04:33:19.256 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.257 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.257 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.257 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:56.326930+0000", 2024-04-26T04:33:19.257 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.7448999999999998e-05 2024-04-26T04:33:19.257 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.257 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.257 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.257 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.257 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.257 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:757 3.8s0 3:16690848:::smithi17829308-32:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e68)", 2024-04-26T04:33:19.257 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:03.326308+0000", 2024-04-26T04:33:19.257 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 15.815864483, 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.93499031499999996, 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 757 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.326308+0000", 2024-04-26T04:33:19.258 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.326308+0000", 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.326311+0000", 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3599999999999999e-06 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.259 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.326322+0000", 2024-04-26T04:33:19.260 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1686e-05 2024-04-26T04:33:19.260 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.260 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.260 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.260 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.326359+0000", 2024-04-26T04:33:19.260 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.6513000000000001e-05 2024-04-26T04:33:19.260 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.260 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.260 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.260 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.326380+0000", 2024-04-26T04:33:19.260 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0806e-05 2024-04-26T04:33:19.260 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.260 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.261 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.261 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.326627+0000", 2024-04-26T04:33:19.261 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000247314 2024-04-26T04:33:19.261 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.261 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.261 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:19.261 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.326643+0000", 2024-04-26T04:33:19.261 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5503e-05 2024-04-26T04:33:19.261 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.261 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.261 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.262 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.430567+0000", 2024-04-26T04:33:19.262 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.103924945 2024-04-26T04:33:19.262 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.262 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.262 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:19.262 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:03.430612+0000", 2024-04-26T04:33:19.262 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.4366000000000002e-05 2024-04-26T04:33:19.262 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.262 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.262 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.262 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:04.260770+0000", 2024-04-26T04:33:19.262 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.83015804100000001 2024-04-26T04:33:19.262 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:04.261016+0000", 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024577699999999997 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:04.261299+0000", 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000283004 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.263 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:904 3.8s0 3:1c568fb8:::smithi17829308-5:head [append 0~770048] snapc 18=[18,17,16,14,11,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:14.317931+0000", 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.8242422759999997, 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69411572399999999, 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 904 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.264 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.317931+0000", 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.317931+0000", 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.317933+0000", 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9359999999999999e-06 2024-04-26T04:33:19.265 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.319177+0000", 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001244013 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.319212+0000", 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4099e-05 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.319229+0000", 2024-04-26T04:33:19.266 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7555999999999999e-05 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.502523+0000", 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.183293759 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.502682+0000", 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000159381 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.267 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.504419+0000", 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0017369670000000001 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.609551+0000", 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10513222999999999 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.011926+0000", 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.40237404100000002 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.268 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.012046+0000", 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000120742 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:905 3.8s0 3:1c568fb8:::smithi17829308-5:head [append 0~540672] snapc 18=[18,17,16,14,11,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:14.327100+0000", 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.8150729290000003, 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.78576857300000003, 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.269 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 905 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.327100+0000", 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.327100+0000", 2024-04-26T04:33:19.270 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.327102+0000", 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3530000000000002e-06 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.327872+0000", 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000769431 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.327904+0000", 2024-04-26T04:33:19.271 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1859000000000001e-05 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.327919+0000", 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5505000000000001e-05 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.606366+0000", 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.27844724500000001 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.272 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.606542+0000", 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000175353 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.607999+0000", 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0014578449999999999 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.810399+0000", 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20239948199999999 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.273 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.112742+0000", 2024-04-26T04:33:19.274 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.302343378 2024-04-26T04:33:19.274 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.274 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.274 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.274 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.112868+0000", 2024-04-26T04:33:19.274 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000126122 2024-04-26T04:33:19.274 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.274 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.274 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.274 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.274 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.274 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:906 3.8s0 3:1c568fb8:::smithi17829308-5:head [append 0~753664] snapc 18=[18,17,16,14,11,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-04-26T04:33:19.274 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:14.338229+0000", 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.8039440339999997, 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.98653850200000004, 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 906 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.275 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.338229+0000", 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.338229+0000", 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.338231+0000", 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4200000000000001e-06 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.276 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.339255+0000", 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0010235050000000001 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.339287+0000", 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2101999999999999e-05 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.339302+0000", 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5642e-05 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.277 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.113637+0000", 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.77433495399999996 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.113814+0000", 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000176477 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.116127+0000", 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0023129800000000001 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.278 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.120934+0000", 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0048073919999999997 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.324556+0000", 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20362176800000001 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.324767+0000", 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021126200000000001 2024-04-26T04:33:19.279 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:907 3.8s0 3:1c568fb8:::smithi17829308-5:head [append 0~262144] snapc 18=[18,17,16,14,11,10] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:14.341563+0000", 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 4.8006101470000004, 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.98390561300000001, 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:19.280 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 907 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.341563+0000", 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.341563+0000", 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.281 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.341565+0000", 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3029999999999998e-06 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.342022+0000", 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00045667599999999999 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.342053+0000", 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1297999999999997e-05 2024-04-26T04:33:19.282 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.342069+0000", 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5773999999999998e-05 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.117555+0000", 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.77548672900000004 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.117852+0000", 2024-04-26T04:33:19.283 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00029663900000000002 2024-04-26T04:33:19.284 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.284 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.284 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.284 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.119601+0000", 2024-04-26T04:33:19.284 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0017485319999999999 2024-04-26T04:33:19.284 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.284 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.284 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.284 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.221244+0000", 2024-04-26T04:33:19.284 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101642892 2024-04-26T04:33:19.284 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.284 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.325260+0000", 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.104016874 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:15.325468+0000", 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000207896 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.285 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:19.286 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-26T04:33:19.289 INFO:teuthology.orchestra.run.smithi012.stderr:marked out osd.10. 2024-04-26T04:33:19.289 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T04:33:19.290 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:19.284+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:19.290 INFO:tasks.rados.rados.0.smithi178.stdout:364: done (3 left) 2024-04-26T04:33:19.290 INFO:tasks.rados.rados.0.smithi178.stdout:365: setattr oid 12 current snap is 23 2024-04-26T04:33:19.290 INFO:tasks.rados.rados.0.smithi178.stdout:353: finishing write tid 6 to smithi17829308-5 2024-04-26T04:33:19.290 INFO:tasks.rados.rados.0.smithi178.stdout:353: oid 5 updating version 0 to 113 2024-04-26T04:33:19.290 INFO:tasks.rados.rados.0.smithi178.stdout:353: oid 5 version 113 is already newer than 111 2024-04-26T04:33:19.290 INFO:tasks.rados.rados.0.smithi178.stdout:353: oid 5 version 113 is already newer than 110 2024-04-26T04:33:19.290 INFO:tasks.rados.rados.0.smithi178.stdout:353: oid 5 updating version 113 to 114 2024-04-26T04:33:19.290 INFO:tasks.rados.rados.0.smithi178.stdout:353: oid 5 version 114 is already newer than 112 2024-04-26T04:33:19.290 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 5 v 114 (ObjNum 138 snap 23 seq_num 138) dirty exists 2024-04-26T04:33:19.290 INFO:tasks.rados.rados.0.smithi178.stdout:353: left oid 5 (ObjNum 138 snap 23 seq_num 138) 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:360: finishing write tid 1 to smithi17829308-25 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:358: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:360: finishing write tid 2 to smithi17829308-25 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:360: finishing write tid 3 to smithi17829308-25 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:360: finishing write tid 4 to smithi17829308-25 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:360: finishing write tid 5 to smithi17829308-25 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:360: finishing write tid 6 to smithi17829308-25 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:360: finishing write tid 7 to smithi17829308-25 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:360: oid 25 updating version 0 to 14 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:360: oid 25 version 14 is already newer than 13 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:360: oid 25 updating version 14 to 15 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:360: oid 25 updating version 15 to 17 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:360: oid 25 updating version 17 to 18 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:360: oid 25 version 18 is already newer than 16 2024-04-26T04:33:19.291 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 25 v 18 (ObjNum 141 snap 23 seq_num 141) dirty exists 2024-04-26T04:33:19.292 INFO:tasks.rados.rados.0.smithi178.stdout:360: left oid 25 (ObjNum 141 snap 23 seq_num 141) 2024-04-26T04:33:19.292 INFO:tasks.rados.rados.0.smithi178.stdout:353: done (3 left) 2024-04-26T04:33:19.292 INFO:tasks.rados.rados.0.smithi178.stdout:358: done (2 left) 2024-04-26T04:33:19.292 INFO:tasks.rados.rados.0.smithi178.stdout:360: done (1 left) 2024-04-26T04:33:19.292 INFO:tasks.rados.rados.0.smithi178.stdout:366: read oid 27 snap -1 2024-04-26T04:33:19.292 INFO:tasks.rados.rados.0.smithi178.stdout:366: expect (ObjNum 87 snap 12 seq_num 87) 2024-04-26T04:33:19.292 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 32 current snap is 23 2024-04-26T04:33:19.292 INFO:tasks.rados.rados.0.smithi178.stdout:367: seq_num 142 ranges {2686976=442368,3129344=720896,3850240=540672,4390912=688128} 2024-04-26T04:33:19.299 INFO:tasks.rados.rados.0.smithi178.stdout:367: writing smithi17829308-32 from 2686976 to 3129344 tid 1 2024-04-26T04:33:19.307 INFO:tasks.rados.rados.0.smithi178.stdout:367: writing smithi17829308-32 from 3129344 to 3850240 tid 2 2024-04-26T04:33:19.313 INFO:tasks.rados.rados.0.smithi178.stdout:367: writing smithi17829308-32 from 3850240 to 4390912 tid 3 2024-04-26T04:33:19.319 INFO:tasks.rados.rados.0.smithi178.stdout:367: writing smithi17829308-32 from 4390912 to 5079040 tid 4 2024-04-26T04:33:19.319 INFO:tasks.rados.rados.0.smithi178.stdout:368: setattr oid 8 current snap is 23 2024-04-26T04:33:19.322 INFO:tasks.rados.rados.0.smithi178.stdout:369: read oid 20 snap -1 2024-04-26T04:33:19.322 INFO:tasks.rados.rados.0.smithi178.stdout:369: expect (ObjNum 122 snap 16 seq_num 122) 2024-04-26T04:33:19.322 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 15 current snap is 23 2024-04-26T04:33:19.322 INFO:tasks.rados.rados.0.smithi178.stdout:370: seq_num 143 ranges {5095424=458752,5554176=475136,6029312=655360,6684672=671744,7356416=147456} 2024-04-26T04:33:19.361 INFO:tasks.rados.rados.0.smithi178.stdout:370: writing smithi17829308-15 from 5095424 to 5554176 tid 1 2024-04-26T04:33:19.363 INFO:tasks.rados.rados.0.smithi178.stdout:370: writing smithi17829308-15 from 5554176 to 6029312 tid 2 2024-04-26T04:33:19.367 INFO:tasks.rados.rados.0.smithi178.stdout:370: writing smithi17829308-15 from 6029312 to 6684672 tid 3 2024-04-26T04:33:19.371 INFO:tasks.rados.rados.0.smithi178.stdout:370: writing smithi17829308-15 from 6684672 to 7356416 tid 4 2024-04-26T04:33:19.372 INFO:tasks.rados.rados.0.smithi178.stdout:370: writing smithi17829308-15 from 7356416 to 7503872 tid 5 2024-04-26T04:33:19.372 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 11 current snap is 23 2024-04-26T04:33:19.372 INFO:tasks.rados.rados.0.smithi178.stdout:371: seq_num 144 ranges {2490368=16384} 2024-04-26T04:33:19.386 INFO:tasks.rados.rados.0.smithi178.stdout:371: writing smithi17829308-11 from 2490368 to 2506752 tid 1 2024-04-26T04:33:19.387 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 45 current snap is 23 2024-04-26T04:33:19.387 INFO:tasks.rados.rados.0.smithi178.stdout:372: seq_num 145 ranges {0=32768} 2024-04-26T04:33:19.387 INFO:tasks.rados.rados.0.smithi178.stdout:372: writing smithi17829308-45 from 0 to 32768 tid 1 2024-04-26T04:33:19.387 INFO:tasks.rados.rados.0.smithi178.stdout:373: delete oid 41 current snap is 23 2024-04-26T04:33:19.388 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:19.384+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:19.462 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:19.463 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:33:19.463 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.463 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "pg_update_log_missing(3.bs6 epoch 84/30 rep_tid 383 entries 84'94 (0'0) error 3:df125fb3:::smithi17829308-41:head by client.4670.0:956 0.000000 -2 [r=-2+0b, r=0+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 83'91)", 2024-04-26T04:33:19.463 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.388538+0000", 2024-04-26T04:33:19.463 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.074771114999999999, 2024-04-26T04:33:19.463 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.074802998999999995, 2024-04-26T04:33:19.463 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.463 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.463 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "no flag points reached", 2024-04-26T04:33:19.464 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.464 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.464 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.464 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.388538+0000", 2024-04-26T04:33:19.464 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.464 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.464 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.464 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.464 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.388538+0000", 2024-04-26T04:33:19.464 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.465 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.465 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.465 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.465 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.388539+0000", 2024-04-26T04:33:19.465 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4e-07 2024-04-26T04:33:19.465 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.465 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.466 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.466 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.388551+0000", 2024-04-26T04:33:19.466 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2285e-05 2024-04-26T04:33:19.466 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.466 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.466 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.466 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.388577+0000", 2024-04-26T04:33:19.466 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5454999999999999e-05 2024-04-26T04:33:19.466 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.467 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.467 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.467 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.467 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-26T04:33:19.467 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 1 2024-04-26T04:33:19.467 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:19.480 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T04:33:19.489 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:19.485+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:19.590 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:19.592+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:19.642 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:19.642 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:33:19.642 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:33:19.642 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:33:19.642 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:19.656 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T04:33:19.673 INFO:tasks.rados.rados.0.smithi178.stdout:366: expect (ObjNum 87 snap 12 seq_num 87) 2024-04-26T04:33:19.692 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:19.691+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:19.783 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:19.784 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:33:19.784 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:33:19.784 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:33:19.784 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.784 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s2 32/30 ECSubWrite(tid=1, reqid=client.4670.0:5, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:19.784 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.312729+0000", 2024-04-26T04:33:19.784 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 54.468613793999999, 2024-04-26T04:33:19.784 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20126603100000001, 2024-04-26T04:33:19.784 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.784 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.784 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.784 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.784 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.785 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.785 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312729+0000", 2024-04-26T04:33:19.785 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.785 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.785 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.785 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.785 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312729+0000", 2024-04-26T04:33:19.785 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.785 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.785 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.785 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.785 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312731+0000", 2024-04-26T04:33:19.785 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9130000000000001e-06 2024-04-26T04:33:19.786 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.786 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.786 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.786 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312766+0000", 2024-04-26T04:33:19.786 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4768000000000003e-05 2024-04-26T04:33:19.786 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.786 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.786 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.786 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312797+0000", 2024-04-26T04:33:19.786 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1201000000000002e-05 2024-04-26T04:33:19.786 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.786 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.787 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.787 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312814+0000", 2024-04-26T04:33:19.787 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7073000000000001e-05 2024-04-26T04:33:19.787 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.787 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.787 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.787 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.413078+0000", 2024-04-26T04:33:19.787 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10026389400000001 2024-04-26T04:33:19.787 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.787 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.413092+0000", 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4188999999999999e-05 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.513794+0000", 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100702298 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.513995+0000", 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000200695 2024-04-26T04:33:19.788 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.789 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.789 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.789 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.789 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.789 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.7s2 35/30 ECSubRead(tid=58, to_read={3:e1054bf1:::smithi17829308-37:head=0,151552,0}, subchunks={3:e1054bf1:::smithi17829308-37:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:33:19.789 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:28.954839+0000", 2024-04-26T04:33:19.789 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 50.826504053999997, 2024-04-26T04:33:19.789 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20146535800000001, 2024-04-26T04:33:19.789 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.789 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.789 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.789 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.789 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954839+0000", 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954839+0000", 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954840+0000", 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.826e-06 2024-04-26T04:33:19.790 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.791 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.791 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.791 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954857+0000", 2024-04-26T04:33:19.791 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6333000000000001e-05 2024-04-26T04:33:19.791 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.791 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.791 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.791 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954890+0000", 2024-04-26T04:33:19.791 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.3594999999999999e-05 2024-04-26T04:33:19.791 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.791 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.791 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.792 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:28.954912+0000", 2024-04-26T04:33:19.792 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1325e-05 2024-04-26T04:33:19.792 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.792 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.792 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.792 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.155338+0000", 2024-04-26T04:33:19.792 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20042677 2024-04-26T04:33:19.792 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.792 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.793 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.793 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.156304+0000", 2024-04-26T04:33:19.793 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00096550900000000003 2024-04-26T04:33:19.793 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.793 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.793 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.793 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.793 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.793 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs6 35/30 ECSubWrite(tid=59, reqid=client.4670.0:176, at_version=0'0, trim_to=0'0, roll_forward_to=35'11))", 2024-04-26T04:33:19.793 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.055093+0000", 2024-04-26T04:33:19.793 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 50.726250067000002, 2024-04-26T04:33:19.793 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201719765, 2024-04-26T04:33:19.793 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.794 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.794 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.794 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.794 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.794 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.794 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055093+0000", 2024-04-26T04:33:19.794 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.794 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.794 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.794 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.794 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055093+0000", 2024-04-26T04:33:19.794 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.794 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055095+0000", 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.227e-06 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055119+0000", 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4329e-05 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.055151+0000", 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1507999999999997e-05 2024-04-26T04:33:19.795 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.155266+0000", 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100114996 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.255601+0000", 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100334886 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.255617+0000", 2024-04-26T04:33:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6446000000000001e-05 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.256718+0000", 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001101524 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.256812+0000", 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.3849000000000002e-05 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.797 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es1 35/30 ECSubWrite(tid=190, reqid=client.4670.0:235, at_version=0'0, trim_to=0'0, roll_forward_to=35'12))", 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:29.421980+0000", 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 50.359362902000001, 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20113165399999999, 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421980+0000", 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.798 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421980+0000", 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.421982+0000", 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2639999999999998e-06 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.422038+0000", 2024-04-26T04:33:19.799 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.6273000000000001e-05 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.422071+0000", 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2437999999999997e-05 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.522157+0000", 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100086417 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.800 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.622323+0000", 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100166121 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.622332+0000", 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.1309999999999995e-06 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.623023+0000", 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000690491 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.801 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.802 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:29.623111+0000", 2024-04-26T04:33:19.802 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.8518999999999997e-05 2024-04-26T04:33:19.802 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.802 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.802 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.802 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.802 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.802 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.1s2 41/30 ECSubWrite(tid=115, reqid=client.4670.0:355, at_version=41'36, trim_to=0'0, roll_forward_to=40'34))", 2024-04-26T04:33:19.802 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.444369+0000", 2024-04-26T04:33:19.802 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 44.336973987999997, 2024-04-26T04:33:19.803 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20274193700000001, 2024-04-26T04:33:19.803 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.803 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.803 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.803 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.803 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.803 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.803 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444369+0000", 2024-04-26T04:33:19.803 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.803 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.803 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.803 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.803 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444369+0000", 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444371+0000", 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9099999999999999e-06 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444630+0000", 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00025897499999999999 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.804 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444655+0000", 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5819000000000001e-05 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.444670+0000", 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4779e-05 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.644608+0000", 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19993833 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.805 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.644623+0000", 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4806e-05 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.647032+0000", 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0024082880000000002 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.647111+0000", 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.9029999999999994e-05 2024-04-26T04:33:19.806 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.7s2 45/30 ECSubWrite(tid=137, reqid=client.4670.0:371, at_version=45'18, trim_to=0'0, roll_forward_to=44'15))", 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.608809+0000", 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 40.172533737000002, 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20229014100000001, 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.807 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608809+0000", 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608809+0000", 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.608811+0000", 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2730000000000001e-06 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.808 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609164+0000", 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00035305999999999998 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609195+0000", 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0818000000000002e-05 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609214+0000", 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8819000000000001e-05 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.809 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.709463+0000", 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10024947300000001 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.709490+0000", 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6312000000000002e-05 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.811034+0000", 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10154439899999999 2024-04-26T04:33:19.810 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.811099+0000", 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.4987000000000006e-05 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.7s2 45/30 ECSubWrite(tid=138, reqid=client.4670.0:372, at_version=45'19, trim_to=0'0, roll_forward_to=44'15))", 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:39.609251+0000", 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 40.172091242, 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20191711300000001, 2024-04-26T04:33:19.811 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609251+0000", 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609251+0000", 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.812 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.813 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609252+0000", 2024-04-26T04:33:19.813 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2999999999999999e-07 2024-04-26T04:33:19.813 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.813 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.813 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.813 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609276+0000", 2024-04-26T04:33:19.813 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3703e-05 2024-04-26T04:33:19.813 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.813 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.813 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.813 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609292+0000", 2024-04-26T04:33:19.813 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6535999999999999e-05 2024-04-26T04:33:19.813 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.609303+0000", 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0539e-05 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.710393+0000", 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101089625 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.710408+0000", 2024-04-26T04:33:19.814 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5079000000000001e-05 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.811107+0000", 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10069903099999999 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:39.811169+0000", 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.177e-05 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.815 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 45/30 ECSubWrite(tid=35, reqid=client.4670.0:377, at_version=0'0, trim_to=0'0, roll_forward_to=45'23))", 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:40.134471+0000", 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 39.646871443000002, 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20268414700000001, 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134471+0000", 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.816 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134471+0000", 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134473+0000", 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2000000000000001e-06 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134501+0000", 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7441999999999999e-05 2024-04-26T04:33:19.817 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.818 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.818 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.818 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.134540+0000", 2024-04-26T04:33:19.818 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.9181999999999999e-05 2024-04-26T04:33:19.818 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.818 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.818 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.818 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234639+0000", 2024-04-26T04:33:19.818 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100099332 2024-04-26T04:33:19.818 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.818 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.818 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.818 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234798+0000", 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015831999999999999 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234822+0000", 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4766e-05 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.337076+0000", 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102253652 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.337155+0000", 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.9252999999999995e-05 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 45/30 ECSubWrite(tid=36, reqid=client.4670.0:379, at_version=45'25, trim_to=0'0, roll_forward_to=45'23))", 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:40.234694+0000", 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 39.546648511999997, 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.2029495, 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.820 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.821 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.821 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.821 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.821 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234694+0000", 2024-04-26T04:33:19.821 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.821 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.821 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.821 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.821 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234694+0000", 2024-04-26T04:33:19.821 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.821 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.821 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.821 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.234695+0000", 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.356e-06 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235050+0000", 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00035439899999999998 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235076+0000", 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5619e-05 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.822 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235096+0000", 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0046000000000001e-05 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.335458+0000", 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10036199799999999 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.335472+0000", 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.472e-05 2024-04-26T04:33:19.823 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.437566+0000", 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102093589 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.437644+0000", 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7773000000000002e-05 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.824 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpRead(3.4s1 45/30 ECSubRead(tid=37, to_read={3:26eb43ca:::smithi17829308-16:head=0,4096,0}, subchunks={3:26eb43ca:::smithi17829308-16:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:40.235130+0000", 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 39.54621289, 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20238550699999999, 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235130+0000", 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.825 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235130+0000", 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235131+0000", 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1739999999999999e-06 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235146+0000", 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5175e-05 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.826 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235172+0000", 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5406e-05 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.235195+0000", 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3266000000000001e-05 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.437356+0000", 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20216114600000001 2024-04-26T04:33:19.827 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:40.437515+0000", 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015934 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 47/30 ECSubWrite(tid=42, reqid=osd.8.0:42, at_version=47'31, trim_to=0'0, roll_forward_to=46'29))", 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:41.248219+0000", 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 38.533123770000003, 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20129562100000001, 2024-04-26T04:33:19.828 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.829 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.829 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.829 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.829 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.829 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.829 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248219+0000", 2024-04-26T04:33:19.829 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.829 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.829 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.829 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.829 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248219+0000", 2024-04-26T04:33:19.829 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.829 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.830 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.830 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.830 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248220+0000", 2024-04-26T04:33:19.830 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5400000000000001e-06 2024-04-26T04:33:19.830 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.830 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.830 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.830 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248271+0000", 2024-04-26T04:33:19.830 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.0377999999999998e-05 2024-04-26T04:33:19.830 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.830 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.830 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.830 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.248305+0000", 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4598999999999999e-05 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348396+0000", 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10009045699999999 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348535+0000", 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000138975 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.831 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.348560+0000", 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5333e-05 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.449434+0000", 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100873729 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:41.449514+0000", 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.0610000000000002e-05 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.832 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.7s2 57/30 ECSubWrite(tid=185, reqid=osd.4.0:185, at_version=57'40, trim_to=0'0, roll_forward_to=54'38))", 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:51.412798+0000", 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 28.368544253, 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201917122, 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.412798+0000", 2024-04-26T04:33:19.833 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.412798+0000", 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.412799+0000", 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.3000000000000001e-07 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.834 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.412836+0000", 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.6808999999999999e-05 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.412865+0000", 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9016000000000001e-05 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.512998+0000", 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100133465 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.835 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.613254+0000", 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10025622300000001 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.613269+0000", 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4946000000000001e-05 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.614645+0000", 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0013753420000000001 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.836 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.837 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:51.614716+0000", 2024-04-26T04:33:19.837 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0790999999999994e-05 2024-04-26T04:33:19.837 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.837 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.837 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.837 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.837 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.837 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.as3 62/30 ECSubWrite(tid=48, reqid=client.4670.0:645, at_version=62'36, trim_to=0'0, roll_forward_to=59'35))", 2024-04-26T04:33:19.837 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:57.510030+0000", 2024-04-26T04:33:19.837 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 22.271313007, 2024-04-26T04:33:19.837 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20227371499999999, 2024-04-26T04:33:19.837 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.838 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.838 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.838 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.838 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.838 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.838 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.510030+0000", 2024-04-26T04:33:19.838 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.838 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.838 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.838 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.838 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.510030+0000", 2024-04-26T04:33:19.838 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.510032+0000", 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2400000000000002e-06 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.510418+0000", 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000386499 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.839 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.510449+0000", 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0335999999999999e-05 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.510466+0000", 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7385999999999999e-05 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.710808+0000", 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20034220799999999 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.840 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.841 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.710824+0000", 2024-04-26T04:33:19.841 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5285999999999999e-05 2024-04-26T04:33:19.841 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.841 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.841 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.841 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.712235+0000", 2024-04-26T04:33:19.841 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0014111480000000001 2024-04-26T04:33:19.841 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.841 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.841 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.841 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:57.712303+0000", 2024-04-26T04:33:19.841 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.8612e-05 2024-04-26T04:33:19.841 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.842 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.842 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.842 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.842 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.842 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs6 71/30 ECSubWrite(tid=299, reqid=client.4670.0:803, at_version=71'55, trim_to=0'0, roll_forward_to=71'54))", 2024-04-26T04:33:19.842 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:07.215560+0000", 2024-04-26T04:33:19.842 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.565783, 2024-04-26T04:33:19.842 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20340592499999999, 2024-04-26T04:33:19.842 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.842 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.842 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.842 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.215560+0000", 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.215560+0000", 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.215562+0000", 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.0159999999999998e-06 2024-04-26T04:33:19.843 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.215811+0000", 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000249745 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.215842+0000", 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0753000000000002e-05 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.215858+0000", 2024-04-26T04:33:19.844 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5395000000000001e-05 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.316113+0000", 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100255784 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.316140+0000", 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6478999999999998e-05 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.845 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.418888+0000", 2024-04-26T04:33:19.846 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10274805300000001 2024-04-26T04:33:19.846 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.846 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.846 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.846 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.418966+0000", 2024-04-26T04:33:19.846 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7700000000000005e-05 2024-04-26T04:33:19.846 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.846 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.846 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.846 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.846 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.846 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs6 71/30 ECSubWrite(tid=301, reqid=client.4670.0:805, at_version=71'57, trim_to=0'0, roll_forward_to=71'54))", 2024-04-26T04:33:19.846 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:07.316683+0000", 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 12.464659683000001, 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201990331, 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.316683+0000", 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.847 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.316683+0000", 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.316684+0000", 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0699999999999996e-07 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.316725+0000", 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1394999999999999e-05 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.848 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.316739+0000", 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3799e-05 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.316749+0000", 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.8030000000000008e-06 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.517715+0000", 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20096667800000001 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.849 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.517732+0000", 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6282000000000001e-05 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.518600+0000", 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00086797999999999997 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:07.518673+0000", 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3687000000000001e-05 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.850 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.7s2 74/30 ECSubWrite(tid=311, reqid=client.4670.0:847, at_version=74'57, trim_to=0'0, roll_forward_to=69'54))", 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:09.543205+0000", 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 10.238137659, 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20299848600000001, 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.851 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:09.543205+0000", 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:09.543205+0000", 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:09.543207+0000", 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2699999999999999e-06 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.852 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:09.543396+0000", 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000188529 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:09.543429+0000", 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2867999999999998e-05 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:09.643607+0000", 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10017859899999999 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.853 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.854 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:09.743899+0000", 2024-04-26T04:33:19.854 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100291617 2024-04-26T04:33:19.854 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.854 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.854 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.854 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:09.743915+0000", 2024-04-26T04:33:19.854 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5906999999999999e-05 2024-04-26T04:33:19.854 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.854 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.854 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.854 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:09.746141+0000", 2024-04-26T04:33:19.854 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0022261210000000002 2024-04-26T04:33:19.854 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:09.746203+0000", 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.2575000000000003e-05 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.4s1 77/30 ECSubWrite(tid=106, reqid=client.4670.0:890, at_version=77'84, trim_to=0'0, roll_forward_to=69'82))", 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:12.500761+0000", 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 7.2805816190000003, 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20183335299999999, 2024-04-26T04:33:19.855 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:12.500761+0000", 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:12.500761+0000", 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.856 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.857 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.857 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:12.500762+0000", 2024-04-26T04:33:19.857 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.1699999999999998e-07 2024-04-26T04:33:19.857 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.857 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.857 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.857 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:12.500813+0000", 2024-04-26T04:33:19.857 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.1014999999999999e-05 2024-04-26T04:33:19.857 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.857 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.857 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.857 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:12.500846+0000", 2024-04-26T04:33:19.857 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.3658999999999997e-05 2024-04-26T04:33:19.858 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.858 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.858 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.858 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:12.600984+0000", 2024-04-26T04:33:19.858 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10013741399999999 2024-04-26T04:33:19.858 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.858 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.858 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.858 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:12.601119+0000", 2024-04-26T04:33:19.858 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013569499999999999 2024-04-26T04:33:19.858 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.858 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.858 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:12.601135+0000", 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5172e-05 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:12.702537+0000", 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101402774 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:12.702594+0000", 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.7006999999999998e-05 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.859 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.860 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.860 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.860 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.860 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.es1 78/30 ECSubWrite(tid=272, reqid=osd.13.0:271, at_version=0'0, trim_to=0'0, roll_forward_to=78'74))", 2024-04-26T04:33:19.860 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:13.879090+0000", 2024-04-26T04:33:19.860 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.9022527980000001, 2024-04-26T04:33:19.860 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.201621886, 2024-04-26T04:33:19.860 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.860 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.860 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.860 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.860 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.861 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.861 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:13.879090+0000", 2024-04-26T04:33:19.861 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.861 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.861 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.861 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.861 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:13.879090+0000", 2024-04-26T04:33:19.861 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.861 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.861 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.861 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.861 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:13.879092+0000", 2024-04-26T04:33:19.861 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1299999999999999e-06 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:13.879121+0000", 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8972e-05 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:13.879159+0000", 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8158999999999998e-05 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:13.979316+0000", 2024-04-26T04:33:19.862 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10015666099999999 2024-04-26T04:33:19.863 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.863 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.863 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.863 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.079588+0000", 2024-04-26T04:33:19.863 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100271887 2024-04-26T04:33:19.863 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.863 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.863 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.863 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.079613+0000", 2024-04-26T04:33:19.863 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5222e-05 2024-04-26T04:33:19.863 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.863 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.863 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.080622+0000", 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001008828 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.080712+0000", 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.0026999999999997e-05 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.bs6 79/30 ECSubWrite(tid=340, reqid=osd.4.0:340, at_version=79'73, trim_to=0'0, roll_forward_to=78'71))", 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:14.348604+0000", 2024-04-26T04:33:19.864 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 5.4327381810000004, 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.202645825, 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.348604+0000", 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.348604+0000", 2024-04-26T04:33:19.865 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.866 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.866 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.866 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.866 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.348605+0000", 2024-04-26T04:33:19.866 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.53e-07 2024-04-26T04:33:19.866 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.866 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.866 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.866 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.348668+0000", 2024-04-26T04:33:19.866 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3027999999999994e-05 2024-04-26T04:33:19.866 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.866 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.866 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.348692+0000", 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4079000000000001e-05 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.348709+0000", 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6699000000000001e-05 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.448980+0000", 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10027102 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.867 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.449006+0000", 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6469e-05 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.551168+0000", 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102161803 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:14.551250+0000", 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2373999999999994e-05 2024-04-26T04:33:19.868 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWrite(3.as5 83/80 ECSubWrite(tid=85, reqid=osd.3.0:85, at_version=83'61, trim_to=0'0, roll_forward_to=82'57))", 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:18.542414+0000", 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.2389282189999999, 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.202143614, 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "reached pg", 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:19.869 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.542414+0000", 2024-04-26T04:33:19.870 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.870 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.870 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.870 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:19.870 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.542414+0000", 2024-04-26T04:33:19.870 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:19.870 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.870 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.870 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:19.870 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.542416+0000", 2024-04-26T04:33:19.870 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.079e-06 2024-04-26T04:33:19.870 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.870 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.542441+0000", 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5248e-05 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.542457+0000", 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.594e-05 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.642558+0000", 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100100827 2024-04-26T04:33:19.871 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.742894+0000", 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100336432 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.742926+0000", 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2221999999999997e-05 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.744448+0000", 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0015216330000000001 2024-04-26T04:33:19.872 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:19.873 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:19.873 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:19.873 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:18.744558+0000", 2024-04-26T04:33:19.873 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000110233 2024-04-26T04:33:19.873 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.873 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.873 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.873 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:19.873 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:19.874 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:19.876 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 8 v 19 (ObjNum 98 snap 12 seq_num 98) dirty exists 2024-04-26T04:33:19.876 INFO:tasks.rados.rados.0.smithi178.stdout:366: done (8 left) 2024-04-26T04:33:19.876 INFO:tasks.rados.rados.0.smithi178.stdout:368: done (7 left) 2024-04-26T04:33:19.876 INFO:tasks.rados.rados.0.smithi178.stdout:373: done (6 left) 2024-04-26T04:33:19.876 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 21 current snap is 23 2024-04-26T04:33:19.876 INFO:tasks.rados.rados.0.smithi178.stdout:374: seq_num 147 ranges {3211264=65536} 2024-04-26T04:33:19.876 INFO:tasks.rados.rados.0.smithi178.stdout:374: writing smithi17829308-21 from 3211264 to 3276800 tid 1 2024-04-26T04:33:19.876 INFO:tasks.rados.rados.0.smithi178.stdout:375: setattr oid 36 current snap is 23 2024-04-26T04:33:19.876 INFO:tasks.rados.rados.0.smithi178.stdout:376: read oid 3 snap -1 2024-04-26T04:33:19.876 INFO:tasks.rados.rados.0.smithi178.stdout:376: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-26T04:33:19.876 INFO:tasks.rados.rados.0.smithi178.stdout:377: delete oid 37 current snap is 23 2024-04-26T04:33:19.877 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T04:33:19.877 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:19.874+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:19.910 INFO:tasks.rados.rados.0.smithi178.stdout:372: finishing write tid 1 to smithi17829308-45 2024-04-26T04:33:19.929 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 12 v 132 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-04-26T04:33:19.929 INFO:tasks.rados.rados.0.smithi178.stdout:367: finishing write tid 1 to smithi17829308-32 2024-04-26T04:33:19.930 INFO:tasks.rados.rados.0.smithi178.stdout:367: finishing write tid 2 to smithi17829308-32 2024-04-26T04:33:19.930 INFO:tasks.rados.rados.0.smithi178.stdout:367: finishing write tid 3 to smithi17829308-32 2024-04-26T04:33:19.930 INFO:tasks.rados.rados.0.smithi178.stdout:367: finishing write tid 4 to smithi17829308-32 2024-04-26T04:33:19.930 INFO:tasks.rados.rados.0.smithi178.stdout:367: finishing write tid 5 to smithi17829308-32 2024-04-26T04:33:19.976 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:19.974+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:20.000 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:20.000 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:33:20.000 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.000 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=293, last_complete=84'102, committed=1, applied=1))", 2024-04-26T04:33:20.000 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.806167+0000", 2024-04-26T04:33:20.000 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.19288053999999999, 2024-04-26T04:33:20.000 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.192898243, 2024-04-26T04:33:20.000 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.000 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.000 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.000 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806167+0000", 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806167+0000", 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806167+0000", 2024-04-26T04:33:20.001 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.7300000000000002e-07 2024-04-26T04:33:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806170+0000", 2024-04-26T04:33:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.1499999999999999e-06 2024-04-26T04:33:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806181+0000", 2024-04-26T04:33:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1277000000000001e-05 2024-04-26T04:33:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.002 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806194+0000", 2024-04-26T04:33:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2483000000000001e-05 2024-04-26T04:33:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=294, last_complete=84'103, committed=1, applied=1))", 2024-04-26T04:33:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.807683+0000", 2024-04-26T04:33:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.191364331, 2024-04-26T04:33:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19141375399999999, 2024-04-26T04:33:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.003 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.807683+0000", 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.807683+0000", 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.004 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.005 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.807684+0000", 2024-04-26T04:33:20.005 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.463e-06 2024-04-26T04:33:20.005 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.005 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.005 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.005 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.807693+0000", 2024-04-26T04:33:20.005 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7260000000000008e-06 2024-04-26T04:33:20.005 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.005 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.005 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.006 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.807720+0000", 2024-04-26T04:33:20.006 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7385999999999998e-05 2024-04-26T04:33:20.006 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.006 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.006 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.006 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907829+0000", 2024-04-26T04:33:20.006 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100108472 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=295, last_complete=84'104, committed=1, applied=1))", 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.907854+0000", 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.091192760999999997, 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.091268879999999997, 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.007 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907854+0000", 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907854+0000", 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907855+0000", 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.4600000000000004e-07 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.008 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907860+0000", 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.9429999999999999e-06 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907870+0000", 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0144e-05 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907877+0000", 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.7209999999999997e-06 2024-04-26T04:33:20.009 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.010 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.010 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.010 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.010 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.010 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:944 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~475136 in=475136b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.010 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.463365+0000", 2024-04-26T04:33:20.010 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.53568195200000002, 2024-04-26T04:33:20.010 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.53579202199999998, 2024-04-26T04:33:20.010 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.010 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.010 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:20.010 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.011 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.011 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.011 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 944 2024-04-26T04:33:20.011 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.011 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.011 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.011 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.011 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.463365+0000", 2024-04-26T04:33:20.011 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.011 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.011 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.011 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.011 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.463365+0000", 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.463367+0000", 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.824e-06 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.464200+0000", 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00083280500000000005 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.012 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.464235+0000", 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.5494999999999998e-05 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.464255+0000", 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9942999999999999e-05 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.467795+0000", 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0035392290000000001 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.013 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.467959+0000", 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016452200000000001 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.469278+0000", 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001318701 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.476423+0000", 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0071450630000000001 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.014 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.015 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.016 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.016 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.016 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=296, last_complete=84'105, committed=1, applied=1))", 2024-04-26T04:33:20.017 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.907884+0000", 2024-04-26T04:33:20.017 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.091163405000000003, 2024-04-26T04:33:20.017 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.091317887, 2024-04-26T04:33:20.017 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.017 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.017 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.017 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.018 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.018 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.018 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907884+0000", 2024-04-26T04:33:20.018 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.018 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.018 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.018 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.018 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907884+0000", 2024-04-26T04:33:20.019 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.019 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.019 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.019 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.019 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907884+0000", 2024-04-26T04:33:20.019 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.2999999999999999e-07 2024-04-26T04:33:20.019 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.019 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.020 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.020 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907886+0000", 2024-04-26T04:33:20.020 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.1160000000000002e-06 2024-04-26T04:33:20.020 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.020 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.020 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.020 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907891+0000", 2024-04-26T04:33:20.020 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.4839999999999998e-06 2024-04-26T04:33:20.021 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.021 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.021 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.021 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907894+0000", 2024-04-26T04:33:20.021 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.5939999999999998e-06 2024-04-26T04:33:20.021 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.021 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.021 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.022 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.022 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.022 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:945 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~655360 in=655360b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.022 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.464296+0000", 2024-04-26T04:33:20.022 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.53475141299999995, 2024-04-26T04:33:20.022 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.53493590800000002, 2024-04-26T04:33:20.022 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.022 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.023 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:20.023 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.023 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.023 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.023 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 945 2024-04-26T04:33:20.023 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.023 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.023 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.023 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.024 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.464296+0000", 2024-04-26T04:33:20.024 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.024 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.024 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.024 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.024 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.464296+0000", 2024-04-26T04:33:20.024 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.024 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.025 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.025 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.025 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.464298+0000", 2024-04-26T04:33:20.025 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.863e-06 2024-04-26T04:33:20.025 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.025 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.025 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.025 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465275+0000", 2024-04-26T04:33:20.026 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00097692700000000005 2024-04-26T04:33:20.026 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.026 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.026 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.026 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465303+0000", 2024-04-26T04:33:20.026 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8849000000000001e-05 2024-04-26T04:33:20.027 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.027 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.027 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.027 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465321+0000", 2024-04-26T04:33:20.027 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7822999999999999e-05 2024-04-26T04:33:20.027 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.027 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.027 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.027 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.470354+0000", 2024-04-26T04:33:20.027 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0050330050000000001 2024-04-26T04:33:20.027 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.027 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.027 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.028 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.470516+0000", 2024-04-26T04:33:20.028 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016219100000000001 2024-04-26T04:33:20.028 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.028 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.028 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.028 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.471939+0000", 2024-04-26T04:33:20.028 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001422708 2024-04-26T04:33:20.029 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.029 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.029 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.029 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.479058+0000", 2024-04-26T04:33:20.029 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0071184530000000003 2024-04-26T04:33:20.029 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.029 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.029 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.029 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.029 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.030 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=297, last_complete=84'106, committed=1, applied=1))", 2024-04-26T04:33:20.030 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.907907+0000", 2024-04-26T04:33:20.030 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.091139919999999999, 2024-04-26T04:33:20.030 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.091365788000000003, 2024-04-26T04:33:20.030 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.030 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.030 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.030 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.030 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.030 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.030 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907907+0000", 2024-04-26T04:33:20.030 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.030 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.031 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.031 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.031 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907907+0000", 2024-04-26T04:33:20.031 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.031 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.031 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.031 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.031 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907908+0000", 2024-04-26T04:33:20.031 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.23e-07 2024-04-26T04:33:20.031 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.031 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.032 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.032 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907909+0000", 2024-04-26T04:33:20.032 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8560000000000001e-06 2024-04-26T04:33:20.032 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.032 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.032 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.032 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907914+0000", 2024-04-26T04:33:20.032 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.6750000000000001e-06 2024-04-26T04:33:20.032 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.034 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.034 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.034 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907918+0000", 2024-04-26T04:33:20.034 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.749e-06 2024-04-26T04:33:20.034 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.034 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.034 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.034 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.034 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.034 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:946 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~671744 in=671744b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.034 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.465351+0000", 2024-04-26T04:33:20.035 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.53369626199999998, 2024-04-26T04:33:20.035 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.53394989599999998, 2024-04-26T04:33:20.035 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.035 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.035 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:20.035 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.035 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.035 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.035 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 946 2024-04-26T04:33:20.035 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.035 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.035 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.035 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.036 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465351+0000", 2024-04-26T04:33:20.036 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.036 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.036 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.036 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.036 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465351+0000", 2024-04-26T04:33:20.036 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.036 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.036 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.036 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.036 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465353+0000", 2024-04-26T04:33:20.036 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.756e-06 2024-04-26T04:33:20.036 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466405+0000", 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001052281 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466424+0000", 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8749e-05 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466440+0000", 2024-04-26T04:33:20.037 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6143000000000001e-05 2024-04-26T04:33:20.038 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.038 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.038 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.038 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.472946+0000", 2024-04-26T04:33:20.038 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0065059540000000004 2024-04-26T04:33:20.038 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.038 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.038 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.038 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.473103+0000", 2024-04-26T04:33:20.038 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000156682 2024-04-26T04:33:20.038 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.038 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.038 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.039 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.475408+0000", 2024-04-26T04:33:20.039 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.002305788 2024-04-26T04:33:20.039 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.039 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.039 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.039 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.580604+0000", 2024-04-26T04:33:20.039 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.105195311 2024-04-26T04:33:20.039 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.039 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.040 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.040 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.040 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.040 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=298, last_complete=84'107, committed=1, applied=1))", 2024-04-26T04:33:20.040 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.907924+0000", 2024-04-26T04:33:20.040 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.091123523999999997, 2024-04-26T04:33:20.040 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.091419391000000003, 2024-04-26T04:33:20.040 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.040 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.040 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.041 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.041 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.041 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.041 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907924+0000", 2024-04-26T04:33:20.041 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.041 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.041 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.041 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.041 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907924+0000", 2024-04-26T04:33:20.041 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.041 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.041 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.041 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907924+0000", 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6500000000000001e-07 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907926+0000", 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.697e-06 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907930+0000", 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.0539999999999996e-06 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.042 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.043 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.043 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.907933+0000", 2024-04-26T04:33:20.043 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.3850000000000002e-06 2024-04-26T04:33:20.043 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.043 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.044 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.044 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.044 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.044 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:947 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~147456 in=147456b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.466485+0000", 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.53256188100000001, 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.53288545399999998, 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 947 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.045 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.046 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466485+0000", 2024-04-26T04:33:20.046 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.046 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.046 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.046 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.046 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466485+0000", 2024-04-26T04:33:20.046 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.046 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.046 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.046 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.046 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466486+0000", 2024-04-26T04:33:20.046 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.057e-06 2024-04-26T04:33:20.046 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466582+0000", 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.6039999999999995e-05 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466597+0000", 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4652e-05 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466606+0000", 2024-04-26T04:33:20.047 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6600000000000001e-06 2024-04-26T04:33:20.048 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.048 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.048 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.048 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.476508+0000", 2024-04-26T04:33:20.048 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0099023310000000003 2024-04-26T04:33:20.048 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.048 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.048 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.048 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.476640+0000", 2024-04-26T04:33:20.048 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013183499999999999 2024-04-26T04:33:20.049 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.049 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.049 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.049 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.477366+0000", 2024-04-26T04:33:20.049 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00072583000000000001 2024-04-26T04:33:20.049 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.049 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.049 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.049 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.680804+0000", 2024-04-26T04:33:20.049 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.203438693 2024-04-26T04:33:20.049 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.049 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.049 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.050 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.050 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.050 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=298, last_complete=84'107, committed=1, applied=1))", 2024-04-26T04:33:20.050 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.908010+0000", 2024-04-26T04:33:20.050 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.091037377000000003, 2024-04-26T04:33:20.050 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.091401377000000006, 2024-04-26T04:33:20.050 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.908010+0000", 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.908010+0000", 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.051 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.908010+0000", 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.4099999999999999e-07 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.908012+0000", 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9190000000000002e-06 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.908022+0000", 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.6530000000000006e-06 2024-04-26T04:33:20.052 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.053 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.053 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.053 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.908027+0000", 2024-04-26T04:33:20.053 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.7790000000000003e-06 2024-04-26T04:33:20.053 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.053 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.053 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.053 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.053 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.053 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:948 3.es0 3:7dd262f9:::smithi17829308-15:head [setxattr _header (57) in=64b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.053 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.466618+0000", 2024-04-26T04:33:20.053 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.53242875899999997, 2024-04-26T04:33:20.053 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.53282126600000002, 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 948 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466618+0000", 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.054 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.055 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.055 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.055 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466618+0000", 2024-04-26T04:33:20.055 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.055 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.055 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.055 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.055 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466619+0000", 2024-04-26T04:33:20.055 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8299999999999996e-07 2024-04-26T04:33:20.055 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.055 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.055 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.056 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466624+0000", 2024-04-26T04:33:20.056 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.707e-06 2024-04-26T04:33:20.056 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.056 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.056 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.056 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466634+0000", 2024-04-26T04:33:20.056 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7499999999999998e-06 2024-04-26T04:33:20.056 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466641+0000", 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2230000000000001e-06 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.477849+0000", 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.011207674000000001 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.478017+0000", 2024-04-26T04:33:20.057 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016833399999999999 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.478704+0000", 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00068663000000000003 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.680807+0000", 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20210388900000001 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.058 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.059 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.059 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=298, last_complete=84'107, committed=1, applied=1))", 2024-04-26T04:33:20.059 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.908124+0000", 2024-04-26T04:33:20.059 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.090923271, 2024-04-26T04:33:20.059 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.091356815999999993, 2024-04-26T04:33:20.059 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.059 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.059 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.059 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.060 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.060 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.060 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.908124+0000", 2024-04-26T04:33:20.060 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.060 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.908124+0000", 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.908124+0000", 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.53e-07 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.908126+0000", 2024-04-26T04:33:20.061 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.714e-06 2024-04-26T04:33:20.062 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.062 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.062 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.062 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.908134+0000", 2024-04-26T04:33:20.062 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8180000000000004e-06 2024-04-26T04:33:20.062 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.062 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.062 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.062 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.908139+0000", 2024-04-26T04:33:20.062 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.8160000000000001e-06 2024-04-26T04:33:20.062 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.062 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.063 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.063 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.063 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.063 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:949 3.es0 3:7dd262f9:::smithi17829308-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.063 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.466651+0000", 2024-04-26T04:33:20.063 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.53239669000000001, 2024-04-26T04:33:20.063 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.53285848400000002, 2024-04-26T04:33:20.063 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.063 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.063 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "waiting for rw locks", 2024-04-26T04:33:20.063 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.063 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.063 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 949 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466651+0000", 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466651+0000", 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.064 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466651+0000", 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.4000000000000001e-07 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466654+0000", 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8899999999999999e-06 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466665+0000", 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.061e-05 2024-04-26T04:33:20.065 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466672+0000", 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3989999999999998e-06 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.579174+0000", 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.112501955 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.579291+0000", 2024-04-26T04:33:20.066 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000116872 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:950 3.es0 3:7950d065:::smithi17829308-11:head [append 0~16384 in=16384b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.466682+0000", 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.53236522100000006, 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.53286447299999995, 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.067 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 950 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466682+0000", 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466682+0000", 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.068 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466683+0000", 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.9599999999999999e-07 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466720+0000", 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.6823000000000003e-05 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466731+0000", 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1073000000000001e-05 2024-04-26T04:33:20.069 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.070 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.070 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.070 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466739+0000", 2024-04-26T04:33:20.070 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.9990000000000008e-06 2024-04-26T04:33:20.070 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.070 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.070 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.070 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.579336+0000", 2024-04-26T04:33:20.070 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.112597602 2024-04-26T04:33:20.070 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.070 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.071 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.071 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.579430+0000", 2024-04-26T04:33:20.071 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.4275000000000001e-05 2024-04-26T04:33:20.071 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.072 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.072 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.072 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.580036+0000", 2024-04-26T04:33:20.072 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00060588899999999997 2024-04-26T04:33:20.072 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.072 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.072 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.072 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.782029+0000", 2024-04-26T04:33:20.072 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20199300100000001 2024-04-26T04:33:20.072 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.072 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.072 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.072 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:951 3.es0 3:7950d065:::smithi17829308-11:head [setxattr _header (57) in=64b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.466749+0000", 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.532297723, 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.53284401400000003, 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 951 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.073 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.074 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.074 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.074 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466749+0000", 2024-04-26T04:33:20.074 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.074 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.074 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.074 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.074 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466749+0000", 2024-04-26T04:33:20.074 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.074 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.074 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.074 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.074 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466750+0000", 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.23e-07 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466763+0000", 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3096e-05 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466773+0000", 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0033e-05 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.075 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.566848+0000", 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100074602 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.680865+0000", 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.114016722 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.680952+0000", 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7211000000000002e-05 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.076 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.077 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.077 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.681281+0000", 2024-04-26T04:33:20.077 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00032938999999999999 2024-04-26T04:33:20.077 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.077 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.077 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.077 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.882251+0000", 2024-04-26T04:33:20.077 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20096946500000001 2024-04-26T04:33:20.077 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.077 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.077 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.077 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.077 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:952 3.es0 3.a60b0a9e (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.566871+0000", 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.43217602500000002, 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.43276377799999999, 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 952 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.078 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.566871+0000", 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.566871+0000", 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.566872+0000", 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.203e-06 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.079 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.566878+0000", 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.2320000000000001e-06 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.566888+0000", 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.893e-06 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.566894+0000", 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.2380000000000002e-06 2024-04-26T04:33:20.080 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.081 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.081 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.081 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.081 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.081 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=294, last_complete=84'103, committed=1, applied=1))", 2024-04-26T04:33:20.081 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.774984+0000", 2024-04-26T04:33:20.081 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.22406358000000001, 2024-04-26T04:33:20.081 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.22468036, 2024-04-26T04:33:20.081 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.081 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.081 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.081 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.081 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.774984+0000", 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.774984+0000", 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.774984+0000", 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8999999999999995e-07 2024-04-26T04:33:20.082 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.774989+0000", 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.3320000000000002e-06 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.774999+0000", 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0528e-05 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775007+0000", 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.3599999999999998e-06 2024-04-26T04:33:20.083 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=295, last_complete=84'104, committed=1, applied=1))", 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.775031+0000", 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.22401576100000001, 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.22465796699999999, 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.084 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.085 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775031+0000", 2024-04-26T04:33:20.085 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.085 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.085 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.085 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.085 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775031+0000", 2024-04-26T04:33:20.085 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.085 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.085 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.085 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.086 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775032+0000", 2024-04-26T04:33:20.086 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.7899999999999999e-07 2024-04-26T04:33:20.086 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.086 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.086 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.086 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775035+0000", 2024-04-26T04:33:20.086 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.9519999999999999e-06 2024-04-26T04:33:20.086 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.086 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.086 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.086 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775043+0000", 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.3990000000000004e-06 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775049+0000", 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.0399999999999998e-06 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=296, last_complete=84'105, committed=1, applied=1))", 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.775082+0000", 2024-04-26T04:33:20.087 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.22396538299999999, 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.22463323199999999, 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775082+0000", 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775082+0000", 2024-04-26T04:33:20.088 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775082+0000", 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.7e-07 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775085+0000", 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.875e-06 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.089 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775095+0000", 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.893e-06 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775101+0000", 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.8490000000000002e-06 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=297, last_complete=84'106, committed=1, applied=1))", 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.775146+0000", 2024-04-26T04:33:20.090 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.22390137399999999, 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.22459462599999999, 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775146+0000", 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775146+0000", 2024-04-26T04:33:20.091 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775146+0000", 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.2699999999999999e-07 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775149+0000", 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8830000000000002e-06 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775158+0000", 2024-04-26T04:33:20.092 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.0080000000000008e-06 2024-04-26T04:33:20.093 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.093 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.093 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.093 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775165+0000", 2024-04-26T04:33:20.093 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3280000000000002e-06 2024-04-26T04:33:20.093 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.093 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.093 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.093 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.093 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.093 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=298, last_complete=84'107, committed=1, applied=1))", 2024-04-26T04:33:20.093 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.775300+0000", 2024-04-26T04:33:20.093 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.22374690899999999, 2024-04-26T04:33:20.094 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.22446545200000001, 2024-04-26T04:33:20.094 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.094 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.094 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.094 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.094 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.094 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.094 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775300+0000", 2024-04-26T04:33:20.094 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.094 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.094 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.094 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.094 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775300+0000", 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775301+0000", 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.2699999999999999e-07 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775305+0000", 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.6109999999999998e-06 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.095 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775317+0000", 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2064e-05 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.775325+0000", 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1559999999999995e-06 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=290, last_complete=84'98, committed=1, applied=1))", 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.804441+0000", 2024-04-26T04:33:20.096 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.19460597499999999, 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19534987000000001, 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.804441+0000", 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.804441+0000", 2024-04-26T04:33:20.097 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.098 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.098 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.098 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.098 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.804444+0000", 2024-04-26T04:33:20.098 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3539999999999998e-06 2024-04-26T04:33:20.098 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.098 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.098 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.098 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.804457+0000", 2024-04-26T04:33:20.098 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3613e-05 2024-04-26T04:33:20.098 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.098 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.098 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.099 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.804489+0000", 2024-04-26T04:33:20.099 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.2205000000000001e-05 2024-04-26T04:33:20.099 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.099 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.099 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.099 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.804508+0000", 2024-04-26T04:33:20.099 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8746e-05 2024-04-26T04:33:20.099 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.099 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.099 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.099 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.099 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:943 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~458752 in=458752b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.362470+0000", 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.63657752999999995, 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.63734921300000003, 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 943 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.100 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.101 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.101 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.362470+0000", 2024-04-26T04:33:20.101 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.101 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.101 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.101 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.101 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.362470+0000", 2024-04-26T04:33:20.101 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.101 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.101 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.101 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.101 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.362472+0000", 2024-04-26T04:33:20.101 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5830000000000001e-06 2024-04-26T04:33:20.102 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.102 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.102 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.102 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.363181+0000", 2024-04-26T04:33:20.102 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00070880699999999999 2024-04-26T04:33:20.102 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.102 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.102 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.102 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.363205+0000", 2024-04-26T04:33:20.102 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3736000000000001e-05 2024-04-26T04:33:20.102 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.102 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.102 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.103 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.463297+0000", 2024-04-26T04:33:20.103 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100091838 2024-04-26T04:33:20.103 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.103 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.103 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.103 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.463461+0000", 2024-04-26T04:33:20.103 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016464100000000001 2024-04-26T04:33:20.103 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.103 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.103 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.103 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.463672+0000", 2024-04-26T04:33:20.103 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021105700000000001 2024-04-26T04:33:20.103 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465404+0000", 2024-04-26T04:33:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0017314170000000001 2024-04-26T04:33:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.476417+0000", 2024-04-26T04:33:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.011012811000000001 2024-04-26T04:33:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.104 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=291, last_complete=84'100, committed=1, applied=1))", 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.806049+0000", 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.19299812599999999, 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19381025499999999, 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806049+0000", 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.105 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.107 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.107 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.107 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806049+0000", 2024-04-26T04:33:20.107 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.107 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.107 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.107 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.107 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806051+0000", 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.42e-06 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806059+0000", 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1729999999999995e-06 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806085+0000", 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6478999999999998e-05 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.108 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806099+0000", 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4143e-05 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "MOSDECSubOpWriteReply(3.es0 84/30 ECSubWriteReply(tid=292, last_complete=84'101, committed=1, applied=1))", 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.806124+0000", 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.19292309199999999, 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.19376042299999999, 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:20.109 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806124+0000", 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806124+0000", 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806125+0000", 2024-04-26T04:33:20.110 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.4000000000000002e-07 2024-04-26T04:33:20.113 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.113 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806128+0000", 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.4470000000000002e-06 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806147+0000", 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8635999999999999e-05 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.806155+0000", 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1159999999999999e-06 2024-04-26T04:33:20.114 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.115 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.115 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.115 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.115 INFO:teuthology.orchestra.run.smithi113.stdout: ], 2024-04-26T04:33:20.115 INFO:teuthology.orchestra.run.smithi113.stdout: "num_ops": 26 2024-04-26T04:33:20.115 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:20.117 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T04:33:20.118 INFO:tasks.rados.rados.0.smithi178.stdout:372: finishing write tid 2 to smithi17829308-45 2024-04-26T04:33:20.118 INFO:tasks.rados.rados.0.smithi178.stdout:367: finishing write tid 6 to smithi17829308-32 2024-04-26T04:33:20.118 INFO:tasks.rados.rados.0.smithi178.stdout:367: oid 32 updating version 0 to 135 2024-04-26T04:33:20.118 INFO:tasks.rados.rados.0.smithi178.stdout:367: oid 32 version 135 is already newer than 134 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:367: oid 32 updating version 135 to 137 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:367: oid 32 version 137 is already newer than 133 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:367: oid 32 version 137 is already newer than 136 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 32 v 137 (ObjNum 142 snap 23 seq_num 142) dirty exists 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:367: left oid 32 (ObjNum 142 snap 23 seq_num 142) 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:365: done (9 left) 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:367: done (8 left) 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:377: done (7 left) 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:378: delete oid 17 current snap is 23 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:370: finishing write tid 1 to smithi17829308-15 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:370: finishing write tid 2 to smithi17829308-15 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:370: finishing write tid 3 to smithi17829308-15 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:370: finishing write tid 4 to smithi17829308-15 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:370: finishing write tid 5 to smithi17829308-15 2024-04-26T04:33:20.119 INFO:tasks.rados.rados.0.smithi178.stdout:370: finishing write tid 6 to smithi17829308-15 2024-04-26T04:33:20.120 INFO:tasks.rados.rados.0.smithi178.stdout:371: finishing write tid 1 to smithi17829308-11 2024-04-26T04:33:20.120 INFO:tasks.rados.rados.0.smithi178.stdout:371: finishing write tid 2 to smithi17829308-11 2024-04-26T04:33:20.120 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:20.115+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:20.183 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is set 2024-04-26T04:33:20.217 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:20.214+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:20.251 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:20.252 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [], 2024-04-26T04:33:20.252 INFO:teuthology.orchestra.run.smithi113.stdout: "complaint_time": 30, 2024-04-26T04:33:20.252 INFO:teuthology.orchestra.run.smithi113.stdout: "num_blocked_ops": 0 2024-04-26T04:33:20.252 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:20.265 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T04:33:20.292 INFO:tasks.rados.rados.0.smithi178.stdout:370: finishing write tid 7 to smithi17829308-15 2024-04-26T04:33:20.292 INFO:tasks.rados.rados.0.smithi178.stdout:370: oid 15 updating version 0 to 101 2024-04-26T04:33:20.292 INFO:tasks.rados.rados.0.smithi178.stdout:370: oid 15 updating version 101 to 102 2024-04-26T04:33:20.292 INFO:tasks.rados.rados.0.smithi178.stdout:370: oid 15 updating version 102 to 104 2024-04-26T04:33:20.292 INFO:tasks.rados.rados.0.smithi178.stdout:370: oid 15 version 104 is already newer than 103 2024-04-26T04:33:20.292 INFO:tasks.rados.rados.0.smithi178.stdout:370: oid 15 updating version 104 to 105 2024-04-26T04:33:20.292 INFO:tasks.rados.rados.0.smithi178.stdout:370: oid 15 version 105 is already newer than 100 2024-04-26T04:33:20.292 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 15 v 105 (ObjNum 143 snap 23 seq_num 143) dirty exists 2024-04-26T04:33:20.292 INFO:tasks.rados.rados.0.smithi178.stdout:370: left oid 15 (ObjNum 143 snap 23 seq_num 143) 2024-04-26T04:33:20.313 INFO:tasks.rados.rados.0.smithi178.stdout:372: finishing write tid 3 to smithi17829308-45 2024-04-26T04:33:20.313 INFO:tasks.rados.rados.0.smithi178.stdout:372: oid 45 updating version 0 to 127 2024-04-26T04:33:20.313 INFO:tasks.rados.rados.0.smithi178.stdout:372: oid 45 version 127 is already newer than 126 2024-04-26T04:33:20.314 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 45 v 127 (ObjNum 145 snap 23 seq_num 145) dirty exists 2024-04-26T04:33:20.314 INFO:tasks.rados.rados.0.smithi178.stdout:372: left oid 45 (ObjNum 145 snap 23 seq_num 145) 2024-04-26T04:33:20.319 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:20.319+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:20.388 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:20.388 INFO:teuthology.orchestra.run.smithi113.stdout: "size": 20, 2024-04-26T04:33:20.388 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 600, 2024-04-26T04:33:20.388 INFO:teuthology.orchestra.run.smithi113.stdout: "ops": [ 2024-04-26T04:33:20.388 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.388 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-26T04:33:20.388 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:33:20.388 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 70.914105079999999, 2024-04-26T04:33:20.388 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.261775152, 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 1 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.389 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.390 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.390 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471104+0000", 2024-04-26T04:33:20.390 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.390 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.390 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.390 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.390 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471107+0000", 2024-04-26T04:33:20.390 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.4540000000000001e-06 2024-04-26T04:33:20.390 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.390 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.390 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.390 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471120+0000", 2024-04-26T04:33:20.390 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3217999999999999e-05 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471147+0000", 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.6461e-05 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.471188+0000", 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.1539000000000003e-05 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.391 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.472842+0000", 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0016540700000000001 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for peered", 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:09.472873+0000", 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0300000000000001e-05 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.729650+0000", 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2567770170000001 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.392 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.729957+0000", 2024-04-26T04:33:20.393 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00030700500000000001 2024-04-26T04:33:20.393 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.393 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.393 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:33:20.393 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.730324+0000", 2024-04-26T04:33:20.393 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00036723099999999998 2024-04-26T04:33:20.393 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.393 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.393 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:33:20.394 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732599+0000", 2024-04-26T04:33:20.394 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0022751260000000001 2024-04-26T04:33:20.394 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.394 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.394 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:33:20.394 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732706+0000", 2024-04-26T04:33:20.394 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010741 2024-04-26T04:33:20.394 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.394 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.394 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.394 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732804+0000", 2024-04-26T04:33:20.394 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7348000000000004e-05 2024-04-26T04:33:20.394 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:10.732880+0000", 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.5972999999999999e-05 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 63.676287148999997, 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60751630199999995, 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 117 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.396 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708922+0000", 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708923+0000", 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3300000000000002e-07 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708929+0000", 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.9959999999999999e-06 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.397 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708941+0000", 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2085e-05 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.708948+0000", 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.8170000000000001e-06 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.820967+0000", 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.112019008 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.398 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821107+0000", 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000139825 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.821309+0000", 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020262899999999999 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:16.822530+0000", 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0012206129999999999 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:33:20.399 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316288+0000", 2024-04-26T04:33:20.400 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.49375772400000001 2024-04-26T04:33:20.400 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.400 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.400 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.400 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316375+0000", 2024-04-26T04:33:20.400 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7312999999999996e-05 2024-04-26T04:33:20.400 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.400 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.400 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.401 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:17.316439+0000", 2024-04-26T04:33:20.401 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3658999999999995e-05 2024-04-26T04:33:20.401 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.401 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.401 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.401 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.401 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.401 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:33:20.401 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:33:20.401 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 61.645703812999997, 2024-04-26T04:33:20.401 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.61583993999999997, 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 164 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.402 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739506+0000", 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.0399999999999995e-07 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739594+0000", 2024-04-26T04:33:20.403 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7046000000000007e-05 2024-04-26T04:33:20.404 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.404 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.404 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.404 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739605+0000", 2024-04-26T04:33:20.404 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1576000000000001e-05 2024-04-26T04:33:20.404 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.404 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739613+0000", 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1459999999999996e-06 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.046981+0000", 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30736738000000002 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.047229+0000", 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00024846999999999998 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.405 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.047619+0000", 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00038963799999999999 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.254219+0000", 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20659982700000001 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355048+0000", 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100829484 2024-04-26T04:33:20.406 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355225+0000", 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00017717399999999999 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355346+0000", 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000120495 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.407 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4263.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 61.645582298000001, 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.71903434899999996, 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4263", 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/2693986622", 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 165 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.408 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739627+0000", 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739628+0000", 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.7000000000000004e-07 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.409 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739645+0000", 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6901999999999998e-05 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739656+0000", 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0556e-05 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:18.739667+0000", 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1535000000000001e-05 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.410 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355474+0000", 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.61580721299999996 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355612+0000", 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000137687 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for subops from 2", 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.355816+0000", 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00020451900000000001 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.411 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "op_commit", 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.357276+0000", 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0014599179999999999 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_commit_rec", 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458392+0000", 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.101115921 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458547+0000", 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015461199999999999 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.412 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.413 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:19.458662+0000", 2024-04-26T04:33:20.413 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000114716 2024-04-26T04:33:20.413 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.413 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.413 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.413 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.413 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.413 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:31 3.es0 3:7950d065:::smithi17829308-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:20.413 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.211346+0000", 2024-04-26T04:33:20.413 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 55.173863150999999, 2024-04-26T04:33:20.413 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80728777299999999, 2024-04-26T04:33:20.413 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.413 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.414 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.414 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.414 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.414 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.414 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 31 2024-04-26T04:33:20.414 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.414 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211346+0000", 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211346+0000", 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211350+0000", 2024-04-26T04:33:20.415 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.3400000000000002e-06 2024-04-26T04:33:20.416 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.416 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.416 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.416 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.211978+0000", 2024-04-26T04:33:20.416 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00062828600000000004 2024-04-26T04:33:20.416 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.416 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.416 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.416 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212008+0000", 2024-04-26T04:33:20.416 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0287999999999999e-05 2024-04-26T04:33:20.416 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.416 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212024+0000", 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5316e-05 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312296+0000", 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100272797 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.312539+0000", 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000242542 2024-04-26T04:33:20.417 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.313858+0000", 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001319312 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319757+0000", 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.005898927 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018515+0000", 2024-04-26T04:33:20.418 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69875785199999996 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018634+0000", 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000119113 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:32 3.es0 3:7950d065:::smithi17829308-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.212058+0000", 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 55.173151142999998, 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80687460799999999, 2024-04-26T04:33:20.419 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 32 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212058+0000", 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.420 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212058+0000", 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212060+0000", 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.2130000000000001e-06 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212065+0000", 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.5260000000000001e-06 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.421 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212084+0000", 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8399000000000001e-05 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.212114+0000", 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.0598000000000001e-05 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.314776+0000", 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10266165200000001 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.422 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.315053+0000", 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00027713999999999999 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.316307+0000", 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00125442 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321719+0000", 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.005411463 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.423 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018816+0000", 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69709755699999998 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.018933+0000", 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00011663999999999999 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.424 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:43 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~245760,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.217138+0000", 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 55.168071103999999, 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80275945999999998, 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 43 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.425 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217138+0000", 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217138+0000", 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217141+0000", 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3499999999999999e-06 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.426 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217563+0000", 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00042241199999999999 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217591+0000", 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.7501999999999999e-05 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217605+0000", 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3866000000000001e-05 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.427 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.317118+0000", 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.099513764000000005 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.317307+0000", 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00018885199999999999 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.318924+0000", 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001616691 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.428 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.321728+0000", 2024-04-26T04:33:20.429 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0028038249999999998 2024-04-26T04:33:20.429 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.429 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.429 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.429 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.019782+0000", 2024-04-26T04:33:20.429 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69805386400000002 2024-04-26T04:33:20.429 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.429 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.429 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.429 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.019898+0000", 2024-04-26T04:33:20.430 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000116334 2024-04-26T04:33:20.430 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.430 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.430 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.430 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.430 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.430 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:44 3.es0 3:7dd262f9:::smithi17829308-15:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:20.430 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:25.217633+0000", 2024-04-26T04:33:20.430 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 55.167576642999997, 2024-04-26T04:33:20.430 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.80280496999999995, 2024-04-26T04:33:20.430 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.430 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.430 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 44 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217633+0000", 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.431 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217633+0000", 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217634+0000", 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.06e-06 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217640+0000", 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.614e-06 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.432 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217654+0000", 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4279e-05 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.217662+0000", 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.2959999999999999e-06 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319858+0000", 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.102196092 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.433 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.319985+0000", 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00012646000000000001 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.320976+0000", 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00099148399999999994 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:25.322066+0000", 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001090246 2024-04-26T04:33:20.434 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.020239+0000", 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.69817257799999999 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:26.020438+0000", 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00019886099999999999 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.435 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:267 3.2s0 3:4497087f:::smithi17829308-22:head [read 0~360448,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.131109+0000", 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 50.254100524999998, 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70452207899999997, 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 267 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.436 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131109+0000", 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131109+0000", 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131111+0000", 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9700000000000002e-06 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.437 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.438 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.438 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131119+0000", 2024-04-26T04:33:20.438 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4600000000000003e-06 2024-04-26T04:33:20.438 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.438 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.438 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.438 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131136+0000", 2024-04-26T04:33:20.438 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6820000000000002e-05 2024-04-26T04:33:20.438 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.438 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.438 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.438 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131146+0000", 2024-04-26T04:33:20.438 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.2499999999999995e-06 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131232+0000", 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6907e-05 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131352+0000", 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000119164 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.835631+0000", 2024-04-26T04:33:20.439 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70427950800000005 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:268 3.2s0 3:4497087f:::smithi17829308-22:head [read 0~360448,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 50.254040979000003, 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70524462600000004, 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.440 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.441 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 268 2024-04-26T04:33:20.441 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.441 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.441 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.441 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.441 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:33:20.441 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.441 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.442 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.442 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.442 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:33:20.442 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.442 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.442 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.442 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.442 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131169+0000", 2024-04-26T04:33:20.442 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.3799999999999997e-07 2024-04-26T04:33:20.442 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.442 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.442 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131173+0000", 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.8929999999999998e-06 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131188+0000", 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.5208000000000001e-05 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131197+0000", 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.1950000000000003e-06 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.443 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.444 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131969+0000", 2024-04-26T04:33:20.444 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00077229900000000005 2024-04-26T04:33:20.444 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.444 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.444 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.444 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.132116+0000", 2024-04-26T04:33:20.444 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014686300000000001 2024-04-26T04:33:20.444 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.444 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.836413+0000", 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70429752999999995 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:269 3.2s0 3:4497087f:::smithi17829308-22:head [read 0~360448,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:30.131236+0000", 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 50.253973088000002, 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.70573893899999995, 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.445 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 269 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131236+0000", 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.446 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.447 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131236+0000", 2024-04-26T04:33:20.447 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.447 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.447 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.448 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.448 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131237+0000", 2024-04-26T04:33:20.448 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 5.9800000000000003e-07 2024-04-26T04:33:20.449 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.449 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.449 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.449 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131241+0000", 2024-04-26T04:33:20.449 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.2239999999999997e-06 2024-04-26T04:33:20.449 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.449 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.449 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.449 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.131250+0000", 2024-04-26T04:33:20.449 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.0629999999999995e-06 2024-04-26T04:33:20.449 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.449 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.450 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.450 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231335+0000", 2024-04-26T04:33:20.450 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.10008484400000001 2024-04-26T04:33:20.450 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.450 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.450 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.450 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231436+0000", 2024-04-26T04:33:20.450 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000100435 2024-04-26T04:33:20.450 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.450 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.450 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.450 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.231576+0000", 2024-04-26T04:33:20.450 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00014048000000000001 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:30.836975+0000", 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.605399295 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:353 3.es0 3:7dd262f9:::smithi17829308-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:32:35.627803+0000", 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 44.757407008000001, 2024-04-26T04:33:20.451 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.66700178399999999, 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "started", 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 353 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627803+0000", 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627803+0000", 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627804+0000", 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.68e-06 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627813+0000", 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.4759999999999998e-06 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.453 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627833+0000", 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9809e-05 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.627844+0000", 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1250000000000001e-05 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.691576+0000", 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.063732043000000002 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.454 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.691730+0000", 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00015389900000000001 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.995375+0000", 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30364495600000002 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:35.995640+0000", 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00026531100000000001 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.455 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:32:36.294804+0000", 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.29916436000000002 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:943 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~458752 in=458752b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.362470+0000", 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 1.0227398999999999, 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.72427090900000002, 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.456 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 943 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.362470+0000", 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.362470+0000", 2024-04-26T04:33:20.457 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.362472+0000", 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.5830000000000001e-06 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.363181+0000", 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00070880699999999999 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.363205+0000", 2024-04-26T04:33:20.458 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.3736000000000001e-05 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.463297+0000", 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100091838 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.463461+0000", 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016464100000000001 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.463672+0000", 2024-04-26T04:33:20.459 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00021105700000000001 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465404+0000", 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0017314170000000001 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.476417+0000", 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.011012811000000001 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.460 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.086632+0000", 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.61021586000000005 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.086741+0000", 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00010815900000000001 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:944 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~475136 in=475136b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.463365+0000", 2024-04-26T04:33:20.461 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.92184432199999999, 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.62363648000000005, 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 944 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.463365+0000", 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.462 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.463 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.463 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.463 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.463365+0000", 2024-04-26T04:33:20.463 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.463 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.463 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.463 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.463 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.463367+0000", 2024-04-26T04:33:20.463 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.824e-06 2024-04-26T04:33:20.463 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.463 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.463 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.464 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.464200+0000", 2024-04-26T04:33:20.464 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00083280500000000005 2024-04-26T04:33:20.464 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.464 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.464 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.464 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.464235+0000", 2024-04-26T04:33:20.464 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.5494999999999998e-05 2024-04-26T04:33:20.464 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.464 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.464 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.464 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.464255+0000", 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.9942999999999999e-05 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.467795+0000", 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0035392290000000001 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.467959+0000", 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016452200000000001 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.465 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.469278+0000", 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001318701 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.476423+0000", 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0071450630000000001 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.086907+0000", 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.61048455400000001 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.466 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.087002+0000", 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.4344000000000003e-05 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:945 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~655360 in=655360b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.464296+0000", 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.92091378300000004, 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.62296054300000003, 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.467 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 945 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.464296+0000", 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.464296+0000", 2024-04-26T04:33:20.468 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.464298+0000", 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.863e-06 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465275+0000", 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00097692700000000005 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.469 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465303+0000", 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 2.8849000000000001e-05 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465321+0000", 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.7822999999999999e-05 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.470354+0000", 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0050330050000000001 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.470 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.470516+0000", 2024-04-26T04:33:20.471 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016219100000000001 2024-04-26T04:33:20.471 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.471 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.471 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.471 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.471939+0000", 2024-04-26T04:33:20.471 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001422708 2024-04-26T04:33:20.471 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.471 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.471 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.471 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.479058+0000", 2024-04-26T04:33:20.471 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0071184530000000003 2024-04-26T04:33:20.471 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.087162+0000", 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.60810390000000003 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.087256+0000", 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.4823999999999999e-05 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.472 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:946 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~671744 in=671744b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.465351+0000", 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.91985863199999995, 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.62234621700000003, 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 946 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.473 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.474 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465351+0000", 2024-04-26T04:33:20.474 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.474 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.474 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.474 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.474 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465351+0000", 2024-04-26T04:33:20.474 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.474 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.474 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.474 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.474 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.465353+0000", 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.756e-06 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466405+0000", 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.001052281 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466424+0000", 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.8749e-05 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.475 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466440+0000", 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.6143000000000001e-05 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.472946+0000", 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0065059540000000004 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.473103+0000", 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000156682 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.476 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.475408+0000", 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.002305788 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.580604+0000", 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.105195311 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.087603+0000", 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.50699899699999995 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.477 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.478 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.087697+0000", 2024-04-26T04:33:20.478 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.4555999999999994e-05 2024-04-26T04:33:20.478 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.478 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.478 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.478 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.478 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.478 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:947 3.es0 3:7dd262f9:::smithi17829308-15:head [append 0~147456 in=147456b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.478 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.466485+0000", 2024-04-26T04:33:20.478 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.91872425099999999, 2024-04-26T04:33:20.478 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.62145253600000006, 2024-04-26T04:33:20.478 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.478 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 947 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466485+0000", 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.479 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466485+0000", 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466486+0000", 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.057e-06 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466582+0000", 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.6039999999999995e-05 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.480 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466597+0000", 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.4652e-05 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466606+0000", 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.6600000000000001e-06 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.476508+0000", 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.0099023310000000003 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.481 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.476640+0000", 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00013183499999999999 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.477366+0000", 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00072583000000000001 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.680804+0000", 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.203438693 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.482 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.087848+0000", 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.40704405399999999 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.087938+0000", 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.9383999999999997e-05 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:948 3.es0 3:7dd262f9:::smithi17829308-15:head [setxattr _header (57) in=64b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.466618+0000", 2024-04-26T04:33:20.483 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.91859112899999995, 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.62159783800000001, 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 948 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466618+0000", 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.484 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.485 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.485 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466618+0000", 2024-04-26T04:33:20.485 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.485 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.485 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.485 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.485 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466619+0000", 2024-04-26T04:33:20.485 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.8299999999999996e-07 2024-04-26T04:33:20.485 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.485 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.485 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.485 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466624+0000", 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 4.707e-06 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466634+0000", 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.7499999999999998e-06 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466641+0000", 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.2230000000000001e-06 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.486 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.477849+0000", 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.011207674000000001 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.478017+0000", 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00016833399999999999 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.478704+0000", 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00068663000000000003 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.487 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.680807+0000", 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20210388900000001 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.088099+0000", 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.40729188199999999 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.088216+0000", 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000116966 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.488 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:950 3.es0 3:7950d065:::smithi17829308-11:head [append 0~16384 in=16384b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.466682+0000", 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.91852759100000003, 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.62271725200000005, 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 950 2024-04-26T04:33:20.489 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466682+0000", 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466682+0000", 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.490 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466683+0000", 2024-04-26T04:33:20.491 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 6.9599999999999999e-07 2024-04-26T04:33:20.491 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.491 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.491 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.491 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466720+0000", 2024-04-26T04:33:20.491 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 3.6823000000000003e-05 2024-04-26T04:33:20.491 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.491 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.491 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.491 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466731+0000", 2024-04-26T04:33:20.491 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.1073000000000001e-05 2024-04-26T04:33:20.491 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.491 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466739+0000", 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.9990000000000008e-06 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.579336+0000", 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.112597602 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.579430+0000", 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 9.4275000000000001e-05 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.492 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.580036+0000", 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00060588899999999997 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.782029+0000", 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20199300100000001 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.089201+0000", 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.30717206699999999 2024-04-26T04:33:20.493 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.089399+0000", 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000197827 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: "description": "osd_op(client.4670.0:951 3.es0 3:7950d065:::smithi17829308-11:head [setxattr _header (57) in=64b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: "initiated_at": "2024-04-26T04:33:19.466749+0000", 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: "age": 0.91846009299999998, 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.62334567399999996, 2024-04-26T04:33:20.494 INFO:teuthology.orchestra.run.smithi113.stdout: "continuous": false, 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: "type_data": { 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: "client_info": { 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: "client": "client.4670", 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: "tid": 951 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: "events": [ 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "initiated", 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466749+0000", 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.495 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "header_read", 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466749+0000", 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "throttled", 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466750+0000", 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 7.23e-07 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "all_read", 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466763+0000", 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.3096e-05 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.496 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "dispatched", 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.466773+0000", 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 1.0033e-05 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.566848+0000", 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.100074602 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "reached_pg", 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.680865+0000", 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.114016722 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.497 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "started", 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.680952+0000", 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 8.7211000000000002e-05 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_started", 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.681281+0000", 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.00032938999999999999 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "sub_op_committed", 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:19.882251+0000", 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20096946500000001 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.498 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.500 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "commit_sent", 2024-04-26T04:33:20.500 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.089963+0000", 2024-04-26T04:33:20.500 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.20771257400000001 2024-04-26T04:33:20.500 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-04-26T04:33:20.500 INFO:teuthology.orchestra.run.smithi113.stdout: { 2024-04-26T04:33:20.500 INFO:teuthology.orchestra.run.smithi113.stdout: "event": "done", 2024-04-26T04:33:20.500 INFO:teuthology.orchestra.run.smithi113.stdout: "time": "2024-04-26T04:33:20.090095+0000", 2024-04-26T04:33:20.500 INFO:teuthology.orchestra.run.smithi113.stdout: "duration": 0.000131858 2024-04-26T04:33:20.500 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.500 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.501 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.501 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-04-26T04:33:20.501 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-04-26T04:33:20.501 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:20.502 INFO:tasks.rados.rados.0.smithi178.stdout:371: finishing write tid 3 to smithi17829308-11 2024-04-26T04:33:20.502 INFO:tasks.rados.rados.0.smithi178.stdout:371: oid 11 updating version 0 to 106 2024-04-26T04:33:20.502 INFO:tasks.rados.rados.0.smithi178.stdout:371: oid 11 updating version 106 to 107 2024-04-26T04:33:20.502 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 11 v 107 (ObjNum 144 snap 23 seq_num 144) dirty exists 2024-04-26T04:33:20.502 INFO:tasks.rados.rados.0.smithi178.stdout:371: left oid 11 (ObjNum 144 snap 23 seq_num 144) 2024-04-26T04:33:20.503 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T04:33:20.503 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:20.500+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:20.603 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:20.603+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:20.636 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:20.636 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:20.636 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:33:20.636 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:20.650 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T04:33:20.704 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:33:20.697+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:33:20.771 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T04:33:20.805 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:20.802+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:20.818 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:20.818 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:20.818 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:33:20.818 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:33:20.818 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:20.834 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T04:33:20.907 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:33:20.906+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:33:20.930 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:33:20.990 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:33:20.991 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:33:20.991 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:33:20.992 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:33:20.992 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:33:20.994 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:20.994 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:33:20.994 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:33:20.994 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:33:20.994 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:20.994 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:42 1.0 e26/23 1:30306672:devicehealth::main.db.0000000000000000:head v 26'28, mlcod=26'26)", 2024-04-26T04:33:20.994 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 67.738760776999996, 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201587136, 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:20.995 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220676+0000", 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.843e-06 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220687+0000", 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1727e-05 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:20.996 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220715+0000", 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8033000000000001e-05 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.320857+0000", 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014171400000001 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.421143+0000", 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100285902 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:20.997 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:20.998 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:20.998 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.421175+0000", 2024-04-26T04:33:20.998 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2413999999999998e-05 2024-04-26T04:33:20.998 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:20.998 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:20.998 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:20.998 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.422168+0000", 2024-04-26T04:33:20.998 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0009931269999999999 2024-04-26T04:33:20.998 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.422261+0000", 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2375999999999998e-05 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:55 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'33, mlcod=27'29)", 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 67.415370913999993, 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.169473288, 2024-04-26T04:33:20.999 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.000 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544065+0000", 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.77e-06 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544195+0000", 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012971599999999999 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544219+0000", 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3476e-05 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.001 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.002 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.002 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.644316+0000", 2024-04-26T04:33:21.002 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100097189 2024-04-26T04:33:21.002 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.002 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.002 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.002 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.711737+0000", 2024-04-26T04:33:21.002 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.067420846000000006 2024-04-26T04:33:21.002 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.002 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.002 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.002 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.711770+0000", 2024-04-26T04:33:21.002 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2959000000000001e-05 2024-04-26T04:33:21.003 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.003 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.003 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.003 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.713428+0000", 2024-04-26T04:33:21.003 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016584130000000001 2024-04-26T04:33:21.003 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.003 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.003 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.003 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.713537+0000", 2024-04-26T04:33:21.003 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010891900000000001 2024-04-26T04:33:21.003 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.003 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.003 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.004 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.004 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.004 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:108 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'70, mlcod=27'64)", 2024-04-26T04:33:21.004 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:33:21.004 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 64.150976751000002, 2024-04-26T04:33:21.004 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30260933299999998, 2024-04-26T04:33:21.004 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.004 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.004 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.004 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.004 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.004 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.37e-07 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.005 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.006 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808464+0000", 2024-04-26T04:33:21.006 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8599999999999998e-06 2024-04-26T04:33:21.006 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.006 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.006 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.006 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808474+0000", 2024-04-26T04:33:21.006 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.9830000000000008e-06 2024-04-26T04:33:21.006 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.006 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.006 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.006 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908648+0000", 2024-04-26T04:33:21.006 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10017390299999999 2024-04-26T04:33:21.006 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908808+0000", 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000159576 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908845+0000", 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7026000000000001e-05 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110983+0000", 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20213835199999999 2024-04-26T04:33:21.007 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.008 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.008 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.008 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111067+0000", 2024-04-26T04:33:21.008 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3996e-05 2024-04-26T04:33:21.008 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.008 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.008 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.008 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.008 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.008 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:109 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'71, mlcod=27'64)", 2024-04-26T04:33:21.008 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:33:21.008 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 64.050742028000002, 2024-04-26T04:33:21.009 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202472819, 2024-04-26T04:33:21.009 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.009 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.009 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.009 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.009 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.009 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.009 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:33:21.009 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.009 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.009 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:33:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908694+0000", 2024-04-26T04:33:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.144e-06 2024-04-26T04:33:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908751+0000", 2024-04-26T04:33:21.010 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7268999999999997e-05 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908769+0000", 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8363e-05 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908780+0000", 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.096e-05 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.909341+0000", 2024-04-26T04:33:21.011 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00056057500000000001 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.909371+0000", 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0357000000000001e-05 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111074+0000", 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20170292500000001 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.012 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111165+0000", 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.1225999999999997e-05 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:110 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'72, mlcod=27'64)", 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 64.050607135999996, 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20385624999999999, 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.013 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.014 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.014 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.014 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:33:21.014 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.014 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.014 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.014 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.014 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:33:21.014 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.014 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.014 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.014 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.015 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908828+0000", 2024-04-26T04:33:21.015 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9400000000000005e-07 2024-04-26T04:33:21.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.015 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.015 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.015 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908836+0000", 2024-04-26T04:33:21.015 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1929999999999993e-06 2024-04-26T04:33:21.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.015 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.015 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908847+0000", 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0703000000000001e-05 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908855+0000", 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7829999999999999e-06 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009054+0000", 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10019966900000001 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.016 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009089+0000", 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4433e-05 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112582+0000", 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103492973 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112684+0000", 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000101802 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.017 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.018 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.018 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.018 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.018 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:111 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'73, mlcod=27'64)", 2024-04-26T04:33:21.018 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:33:21.018 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 64.050568205999994, 2024-04-26T04:33:21.018 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.205422306, 2024-04-26T04:33:21.018 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.018 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.018 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.018 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.018 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.018 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.019 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:33:21.019 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.019 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.019 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.019 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.019 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:33:21.019 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.019 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.019 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.019 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908867+0000", 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.03e-07 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908871+0000", 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5800000000000002e-06 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908883+0000", 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1507e-05 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.020 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009022+0000", 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100139593 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.109703+0000", 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100680741 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.109738+0000", 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5296000000000002e-05 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.021 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.022 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.022 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114207+0000", 2024-04-26T04:33:21.022 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0044682239999999998 2024-04-26T04:33:21.022 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.022 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.022 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.022 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114289+0000", 2024-04-26T04:33:21.022 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2062000000000001e-05 2024-04-26T04:33:21.022 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.022 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.022 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.022 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:112 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'74, mlcod=27'64)", 2024-04-26T04:33:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:33:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 63.950374084000003, 2024-04-26T04:33:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105317487, 2024-04-26T04:33:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.023 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:33:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.024 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.025 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.025 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.025 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:33:21.025 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.025 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.025 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.025 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.025 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009061+0000", 2024-04-26T04:33:21.025 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.057e-06 2024-04-26T04:33:21.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.026 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.026 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.026 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009128+0000", 2024-04-26T04:33:21.026 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6859999999999993e-05 2024-04-26T04:33:21.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.026 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.026 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.027 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009147+0000", 2024-04-26T04:33:21.027 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8196e-05 2024-04-26T04:33:21.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.027 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.027 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009157+0000", 2024-04-26T04:33:21.028 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0743e-05 2024-04-26T04:33:21.028 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.028 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.028 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.028 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110254+0000", 2024-04-26T04:33:21.028 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101096941 2024-04-26T04:33:21.028 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.028 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.028 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.029 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110311+0000", 2024-04-26T04:33:21.029 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6718999999999997e-05 2024-04-26T04:33:21.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.029 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.029 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.029 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114296+0000", 2024-04-26T04:33:21.029 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0039847709999999998 2024-04-26T04:33:21.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.029 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.029 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.029 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114378+0000", 2024-04-26T04:33:21.029 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2200000000000006e-05 2024-04-26T04:33:21.029 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.030 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.030 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.030 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.030 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.030 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:114 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'76, mlcod=27'64)", 2024-04-26T04:33:21.030 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:33:21.030 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 63.950224835, 2024-04-26T04:33:21.030 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105329463, 2024-04-26T04:33:21.030 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.030 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.031 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.032 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:33:21.032 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.53e-07 2024-04-26T04:33:21.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.032 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.032 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.032 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009215+0000", 2024-04-26T04:33:21.032 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7099999999999998e-06 2024-04-26T04:33:21.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.032 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.032 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.032 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009223+0000", 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5599999999999994e-06 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009229+0000", 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0800000000000002e-06 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111865+0000", 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102636071 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.033 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111941+0000", 2024-04-26T04:33:21.034 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5615e-05 2024-04-26T04:33:21.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.034 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.034 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.034 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114468+0000", 2024-04-26T04:33:21.034 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002526875 2024-04-26T04:33:21.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.034 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.035 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.035 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114539+0000", 2024-04-26T04:33:21.035 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1198999999999994e-05 2024-04-26T04:33:21.035 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.035 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.035 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.036 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.036 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:115 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'77, mlcod=27'64)", 2024-04-26T04:33:21.036 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:33:21.036 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 63.950196962, 2024-04-26T04:33:21.036 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105390922, 2024-04-26T04:33:21.036 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.037 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.037 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.038 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.038 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.038 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.038 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:33:21.038 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.038 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.038 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.038 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.34e-07 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009293+0000", 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4954999999999998e-05 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.039 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.040 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009304+0000", 2024-04-26T04:33:21.040 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1263999999999999e-05 2024-04-26T04:33:21.040 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.040 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.040 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.040 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009311+0000", 2024-04-26T04:33:21.040 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6630000000000004e-06 2024-04-26T04:33:21.040 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.040 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.040 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112802+0000", 2024-04-26T04:33:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10349130600000001 2024-04-26T04:33:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112843+0000", 2024-04-26T04:33:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0472999999999997e-05 2024-04-26T04:33:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114547+0000", 2024-04-26T04:33:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001704552 2024-04-26T04:33:21.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.042 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.042 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.042 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114628+0000", 2024-04-26T04:33:21.042 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1374999999999999e-05 2024-04-26T04:33:21.042 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.042 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.042 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.042 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.042 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.042 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:116 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'78, mlcod=27'64)", 2024-04-26T04:33:21.042 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:33:21.042 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 63.950113709, 2024-04-26T04:33:21.042 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105877241, 2024-04-26T04:33:21.043 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.043 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.043 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.043 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.043 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.043 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.043 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:33:21.043 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:33:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:33:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.46e-07 2024-04-26T04:33:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.044 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009326+0000", 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5040000000000004e-06 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009334+0000", 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1929999999999993e-06 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009340+0000", 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.3729999999999998e-06 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.046 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.046 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.113595+0000", 2024-04-26T04:33:21.046 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104254889 2024-04-26T04:33:21.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.046 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.046 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.046 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.113649+0000", 2024-04-26T04:33:21.046 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3887000000000001e-05 2024-04-26T04:33:21.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.046 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.046 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.047 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115105+0000", 2024-04-26T04:33:21.047 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014560969999999999 2024-04-26T04:33:21.047 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.047 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.047 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.047 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115198+0000", 2024-04-26T04:33:21.047 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2851999999999998e-05 2024-04-26T04:33:21.048 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.048 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.048 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.048 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.048 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:117 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'79, mlcod=27'64)", 2024-04-26T04:33:21.048 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:33:21.048 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 63.950086226000003, 2024-04-26T04:33:21.048 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10639752, 2024-04-26T04:33:21.048 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.049 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.050 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.050 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009349+0000", 2024-04-26T04:33:21.050 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7300000000000002e-07 2024-04-26T04:33:21.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.050 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.050 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.051 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009353+0000", 2024-04-26T04:33:21.051 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5970000000000002e-06 2024-04-26T04:33:21.051 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.051 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.051 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.051 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009361+0000", 2024-04-26T04:33:21.052 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0600000000000008e-06 2024-04-26T04:33:21.052 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.052 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.052 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.052 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009381+0000", 2024-04-26T04:33:21.052 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9533000000000001e-05 2024-04-26T04:33:21.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.053 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.053 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.053 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114709+0000", 2024-04-26T04:33:21.053 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105328753 2024-04-26T04:33:21.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.053 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.053 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.053 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114745+0000", 2024-04-26T04:33:21.054 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5976000000000002e-05 2024-04-26T04:33:21.054 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.054 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.054 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.054 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115683+0000", 2024-04-26T04:33:21.054 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00093774899999999998 2024-04-26T04:33:21.054 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.054 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.054 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.054 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115746+0000", 2024-04-26T04:33:21.054 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2578999999999998e-05 2024-04-26T04:33:21.054 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.055 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.055 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.055 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.055 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:155 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'109, mlcod=27'105)", 2024-04-26T04:33:21.056 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:33:21.056 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 62.119898452999998, 2024-04-26T04:33:21.056 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20562082700000001, 2024-04-26T04:33:21.056 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.056 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.056 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.057 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.057 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.057 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:33:21.057 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.057 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.057 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.057 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:33:21.057 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.057 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.058 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.058 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839537+0000", 2024-04-26T04:33:21.058 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7400000000000003e-07 2024-04-26T04:33:21.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.058 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.058 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.058 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839638+0000", 2024-04-26T04:33:21.059 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000101365 2024-04-26T04:33:21.059 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.059 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.059 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.059 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839653+0000", 2024-04-26T04:33:21.059 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4874e-05 2024-04-26T04:33:21.060 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.060 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.060 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.060 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939797+0000", 2024-04-26T04:33:21.060 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014414100000001 2024-04-26T04:33:21.060 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.060 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.061 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.061 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.040092+0000", 2024-04-26T04:33:21.061 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100295305 2024-04-26T04:33:21.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.061 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.061 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.061 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.040151+0000", 2024-04-26T04:33:21.061 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8295999999999999e-05 2024-04-26T04:33:21.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.062 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.062 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.062 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045057+0000", 2024-04-26T04:33:21.062 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0049061390000000003 2024-04-26T04:33:21.062 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.062 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.063 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.063 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045157+0000", 2024-04-26T04:33:21.063 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000100133 2024-04-26T04:33:21.063 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.063 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.063 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.064 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.064 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.064 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:156 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'110, mlcod=27'105)", 2024-04-26T04:33:21.064 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:33:21.064 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 62.019602976000002, 2024-04-26T04:33:21.065 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10750937100000001, 2024-04-26T04:33:21.065 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.065 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.065 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.065 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.065 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.066 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.066 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:33:21.066 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.066 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.066 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.066 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.067 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:33:21.067 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.067 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.067 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.067 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.068 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939833+0000", 2024-04-26T04:33:21.068 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1030000000000001e-06 2024-04-26T04:33:21.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.068 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.068 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.068 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939849+0000", 2024-04-26T04:33:21.069 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6005999999999999e-05 2024-04-26T04:33:21.069 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.069 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.069 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.069 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939866+0000", 2024-04-26T04:33:21.069 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7513e-05 2024-04-26T04:33:21.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.070 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.070 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.070 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939876+0000", 2024-04-26T04:33:21.070 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0154e-05 2024-04-26T04:33:21.071 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.071 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.071 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.071 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.041330+0000", 2024-04-26T04:33:21.071 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101453864 2024-04-26T04:33:21.072 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.072 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.072 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.072 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.041398+0000", 2024-04-26T04:33:21.072 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7617e-05 2024-04-26T04:33:21.072 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.073 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.073 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.073 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047245+0000", 2024-04-26T04:33:21.073 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058470140000000002 2024-04-26T04:33:21.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.074 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.074 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.074 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047341+0000", 2024-04-26T04:33:21.074 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6100000000000005e-05 2024-04-26T04:33:21.074 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.074 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.075 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.075 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.075 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:157 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'111, mlcod=27'105)", 2024-04-26T04:33:21.075 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:33:21.075 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 62.019502836999997, 2024-04-26T04:33:21.076 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107487887, 2024-04-26T04:33:21.076 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.076 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.076 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.076 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.077 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.077 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.077 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:33:21.077 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.077 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.078 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.078 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:33:21.078 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.078 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.079 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:33:21.079 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2300000000000001e-07 2024-04-26T04:33:21.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.079 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.080 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939942+0000", 2024-04-26T04:33:21.080 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4369999999999996e-06 2024-04-26T04:33:21.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.080 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.080 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.080 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939952+0000", 2024-04-26T04:33:21.081 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.075e-05 2024-04-26T04:33:21.081 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.081 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.081 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.081 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939963+0000", 2024-04-26T04:33:21.082 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0069e-05 2024-04-26T04:33:21.082 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.082 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.082 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.082 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.042203+0000", 2024-04-26T04:33:21.082 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102240215 2024-04-26T04:33:21.083 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.083 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.083 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.083 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.042259+0000", 2024-04-26T04:33:21.083 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5955999999999998e-05 2024-04-26T04:33:21.084 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.084 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.084 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.084 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047353+0000", 2024-04-26T04:33:21.084 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0050939080000000003 2024-04-26T04:33:21.084 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.085 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.085 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.085 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047420+0000", 2024-04-26T04:33:21.085 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6928999999999996e-05 2024-04-26T04:33:21.085 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.085 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.086 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.086 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.086 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.086 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:158 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'112, mlcod=27'105)", 2024-04-26T04:33:21.086 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:33:21.087 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 62.019460250999998, 2024-04-26T04:33:21.087 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.209867842, 2024-04-26T04:33:21.087 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.087 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.087 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.087 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.088 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.088 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.088 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:33:21.088 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.088 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.089 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.089 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.089 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:33:21.089 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.090 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.090 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.090 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939975+0000", 2024-04-26T04:33:21.090 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8599999999999999e-07 2024-04-26T04:33:21.090 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.090 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.091 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.091 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940084+0000", 2024-04-26T04:33:21.091 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000109746 2024-04-26T04:33:21.091 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.091 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.091 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.092 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940099+0000", 2024-04-26T04:33:21.092 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4392999999999999e-05 2024-04-26T04:33:21.092 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.092 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.092 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.092 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940110+0000", 2024-04-26T04:33:21.093 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1236999999999999e-05 2024-04-26T04:33:21.093 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.093 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.093 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.093 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043002+0000", 2024-04-26T04:33:21.093 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102892262 2024-04-26T04:33:21.094 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.094 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.094 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.094 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043054+0000", 2024-04-26T04:33:21.094 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1922999999999997e-05 2024-04-26T04:33:21.095 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.095 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.095 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.095 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149774+0000", 2024-04-26T04:33:21.095 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10672018599999999 2024-04-26T04:33:21.095 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149842+0000", 2024-04-26T04:33:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7708999999999998e-05 2024-04-26T04:33:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.096 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:159 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'113, mlcod=27'105)", 2024-04-26T04:33:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:33:21.097 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 62.019311629000001, 2024-04-26T04:33:21.098 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20976125000000001, 2024-04-26T04:33:21.098 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.098 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.098 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.098 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.099 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.099 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.099 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:33:21.099 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.099 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.099 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.100 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.100 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:33:21.100 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.100 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.100 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.100 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.101 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:33:21.101 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7000000000000005e-07 2024-04-26T04:33:21.101 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.101 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.101 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.101 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940131+0000", 2024-04-26T04:33:21.102 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6369999999999999e-06 2024-04-26T04:33:21.102 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.102 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.102 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.102 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940143+0000", 2024-04-26T04:33:21.102 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1603000000000001e-05 2024-04-26T04:33:21.103 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.103 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.103 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.103 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940165+0000", 2024-04-26T04:33:21.103 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2042999999999999e-05 2024-04-26T04:33:21.103 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.103 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.103 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.103 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043777+0000", 2024-04-26T04:33:21.104 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103612145 2024-04-26T04:33:21.104 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.104 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.104 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.104 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043807+0000", 2024-04-26T04:33:21.104 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0562999999999999e-05 2024-04-26T04:33:21.104 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.104 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.104 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.104 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149845+0000", 2024-04-26T04:33:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10603802900000001 2024-04-26T04:33:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149884+0000", 2024-04-26T04:33:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8659999999999999e-05 2024-04-26T04:33:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:160 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'114, mlcod=27'105)", 2024-04-26T04:33:21.105 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:33:21.106 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 62.019261123, 2024-04-26T04:33:21.106 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20974973299999999, 2024-04-26T04:33:21.106 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.106 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.106 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.106 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.106 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.106 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.106 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:33:21.106 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.106 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.106 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.106 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.107 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:33:21.107 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.107 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.107 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.107 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.107 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940174+0000", 2024-04-26T04:33:21.107 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8999999999999998e-07 2024-04-26T04:33:21.107 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.107 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.107 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.107 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940183+0000", 2024-04-26T04:33:21.107 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6970000000000005e-06 2024-04-26T04:33:21.108 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.108 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.108 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.108 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940194+0000", 2024-04-26T04:33:21.108 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0983e-05 2024-04-26T04:33:21.108 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.108 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.108 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940201+0000", 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4400000000000002e-06 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.044326+0000", 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104124984 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.044377+0000", 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1249999999999999e-05 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.109 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149886+0000", 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10550875999999999 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149923+0000", 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7329000000000001e-05 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.110 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:161 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'115, mlcod=27'105)", 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 62.019225179999999, 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.209755314, 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.111 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940210+0000", 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.67e-07 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940330+0000", 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012061900000000001 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.112 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.113 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940344+0000", 2024-04-26T04:33:21.113 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.367e-05 2024-04-26T04:33:21.113 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.113 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.113 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.113 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940355+0000", 2024-04-26T04:33:21.113 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.083e-05 2024-04-26T04:33:21.113 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.113 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.113 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.114 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045239+0000", 2024-04-26T04:33:21.114 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104883873 2024-04-26T04:33:21.114 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.114 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.114 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.114 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045278+0000", 2024-04-26T04:33:21.114 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9904000000000003e-05 2024-04-26T04:33:21.114 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149925+0000", 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104646221 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149965+0000", 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9929999999999999e-05 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.115 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.116 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:162 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'116, mlcod=27'105)", 2024-04-26T04:33:21.116 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:33:21.116 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 62.015761447999999, 2024-04-26T04:33:21.116 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.206336451, 2024-04-26T04:33:21.116 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.116 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.117 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.117 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.117 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.117 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.117 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:33:21.117 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.118 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.118 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.118 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.118 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:33:21.118 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.118 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.118 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.119 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.119 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943675+0000", 2024-04-26T04:33:21.119 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8640000000000001e-06 2024-04-26T04:33:21.119 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.119 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.119 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.119 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943692+0000", 2024-04-26T04:33:21.119 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7503000000000002e-05 2024-04-26T04:33:21.119 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.119 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.119 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.119 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943720+0000", 2024-04-26T04:33:21.119 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7569999999999999e-05 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943734+0000", 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3596e-05 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.046517+0000", 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102783897 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.120 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.046569+0000", 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1706000000000003e-05 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149968+0000", 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10339902199999999 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150009+0000", 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1292999999999999e-05 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.121 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:163 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'117, mlcod=27'105)", 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 62.014260378000003, 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20488266699999999, 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:33:21.122 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945176+0000", 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8470000000000001e-06 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945193+0000", 2024-04-26T04:33:21.123 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7013000000000001e-05 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945221+0000", 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7670000000000001e-05 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945238+0000", 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7113e-05 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:21.124 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.147614+0000", 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202375893 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.147669+0000", 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5729000000000001e-05 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150011+0000", 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002342083 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:21.125 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150057+0000", 2024-04-26T04:33:21.126 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5318999999999998e-05 2024-04-26T04:33:21.126 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.126 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.126 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.126 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:21.126 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:21.126 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:21.126 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T04:33:21.128 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:33:21.128 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:33:21.128 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:33:21.129 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:33:21.129 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:33:21.129 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:33:21.129 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:33:21.130 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:33:21.130 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:33:21.130 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:33:21.131 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:33:21.131 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:33:21.131 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:33:21.131 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:33:21.132 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:21.127+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:21.152 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:33:21.152 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:33:21.176 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:33:21.176 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:33:21.201 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:33:21.201 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:33:21.224 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:33:21.224 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:33:21.229 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:21.226+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:21.247 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:33:21.248 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:33:21.259 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:21.259 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:21.272 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:33:21.272 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:33:21.273 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T04:33:21.330 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:33:21.325+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:33:21.397 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:21.397 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:21.411 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T04:33:21.457 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:21.459+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:21.533 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:21.531+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:21.536 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:21.536 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:21.551 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T04:33:21.634 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:21.629+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:21.700 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 36 v 135 (ObjNum 132 snap 21 seq_num 132) dirty exists 2024-04-26T04:33:21.717 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:21.717 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:21.732 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T04:33:21.734 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:21.730+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:21.836 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:21.834+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:21.853 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:21.853 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:21.867 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T04:33:21.913 INFO:tasks.rados.rados.0.smithi178.stdout:374: finishing write tid 1 to smithi17829308-21 2024-04-26T04:33:21.928 INFO:tasks.rados.rados.0.smithi178.stdout:369: expect (ObjNum 122 snap 16 seq_num 122) 2024-04-26T04:33:21.936 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:21.932+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:21.990 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:21.990 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:22.004 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T04:33:22.037 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:22.036+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:22.099 INFO:tasks.rados.rados.0.smithi178.stdout:374: finishing write tid 2 to smithi17829308-21 2024-04-26T04:33:22.102 INFO:tasks.rados.rados.0.smithi178.stdout:369: done (7 left) 2024-04-26T04:33:22.102 INFO:tasks.rados.rados.0.smithi178.stdout:370: done (6 left) 2024-04-26T04:33:22.102 INFO:tasks.rados.rados.0.smithi178.stdout:371: done (5 left) 2024-04-26T04:33:22.102 INFO:tasks.rados.rados.0.smithi178.stdout:372: done (4 left) 2024-04-26T04:33:22.102 INFO:tasks.rados.rados.0.smithi178.stdout:375: done (3 left) 2024-04-26T04:33:22.102 INFO:tasks.rados.rados.0.smithi178.stdout:378: done (2 left) 2024-04-26T04:33:22.102 INFO:tasks.rados.rados.0.smithi178.stdout:379: snap_create 2024-04-26T04:33:22.127 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:22.128 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:22.128 INFO:tasks.rados.rados.0.smithi178.stdout:376: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-26T04:33:22.138 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:33:22.133+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:33:22.142 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T04:33:22.212 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-26T04:33:22.238 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:22.239+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:22.259 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:22.259 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:22.273 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T04:33:22.339 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:22.334+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:22.350 INFO:tasks.rados.rados.0.smithi178.stdout:376: done (2 left) 2024-04-26T04:33:22.350 INFO:tasks.rados.rados.0.smithi178.stdout:379: done (1 left) 2024-04-26T04:33:22.350 INFO:tasks.rados.rados.0.smithi178.stdout:380: read oid 7 snap -1 2024-04-26T04:33:22.350 INFO:tasks.rados.rados.0.smithi178.stdout:380: expect deleted 2024-04-26T04:33:22.350 INFO:tasks.rados.rados.0.smithi178.stdout:381: read oid 39 snap 23 2024-04-26T04:33:22.350 INFO:tasks.rados.rados.0.smithi178.stdout:381: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:33:22.350 INFO:tasks.rados.rados.0.smithi178.stdout:382: rollback oid 41 current snap is 24 2024-04-26T04:33:22.350 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 41 to 23 2024-04-26T04:33:22.350 INFO:tasks.rados.rados.0.smithi178.stdout:383: rollback oid 5 current snap is 24 2024-04-26T04:33:22.350 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 5 to 20 2024-04-26T04:33:22.350 INFO:tasks.rados.rados.0.smithi178.stdout:384: snap_create 2024-04-26T04:33:22.440 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:22.439+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:22.444 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:22.445 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:22.458 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T04:33:22.508 INFO:tasks.rados.rados.0.smithi178.stdout:374: finishing write tid 3 to smithi17829308-21 2024-04-26T04:33:22.509 INFO:tasks.rados.rados.0.smithi178.stdout:374: oid 21 updating version 0 to 96 2024-04-26T04:33:22.509 INFO:tasks.rados.rados.0.smithi178.stdout:374: oid 21 updating version 96 to 97 2024-04-26T04:33:22.509 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 21 v 97 (ObjNum 147 snap 23 seq_num 147) dirty exists 2024-04-26T04:33:22.509 INFO:tasks.rados.rados.0.smithi178.stdout:374: left oid 21 (ObjNum 147 snap 23 seq_num 147) 2024-04-26T04:33:22.540 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:22.538+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:22.589 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:22.590 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:22.604 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T04:33:22.647 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:33:22.646+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:33:22.653 INFO:tasks.rados.rados.0.smithi178.stdout:382: finishing rollback tid 1 to smithi17829308-41 2024-04-26T04:33:22.653 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 41 v 0 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-04-26T04:33:22.658 INFO:tasks.rados.rados.0.smithi178.stdout:383: finishing rollback tid 0 to smithi17829308-5 2024-04-26T04:33:22.659 INFO:tasks.rados.rados.0.smithi178.stdout:383: finishing rollback tid 1 to smithi17829308-5 2024-04-26T04:33:22.659 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 5 v 140 (ObjNum 33 snap 0 seq_num 2234745376) dirty dne 2024-04-26T04:33:22.743 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:22.742+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:22.758 INFO:tasks.rados.rados.0.smithi178.stdout:381: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-26T04:33:22.767 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:22.767 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:22.783 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T04:33:22.844 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:22.837+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:22.945 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:22.943+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:22.951 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:22.951 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:22.966 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T04:33:23.046 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:23.045+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:23.085 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:23.086 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:23.099 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T04:33:23.147 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:23.147+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:23.225 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:23.225 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:23.240 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T04:33:23.248 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:23.247+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:23.294 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T04:33:23.349 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:23.345+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:23.352 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is set 2024-04-26T04:33:23.358 INFO:tasks.rados.rados.0.smithi178.stdout:374: done (5 left) 2024-04-26T04:33:23.358 INFO:tasks.rados.rados.0.smithi178.stdout:380: done (4 left) 2024-04-26T04:33:23.358 INFO:tasks.rados.rados.0.smithi178.stdout:381: done (3 left) 2024-04-26T04:33:23.358 INFO:tasks.rados.rados.0.smithi178.stdout:382: done (2 left) 2024-04-26T04:33:23.358 INFO:tasks.rados.rados.0.smithi178.stdout:383: done (1 left) 2024-04-26T04:33:23.358 INFO:tasks.rados.rados.0.smithi178.stdout:384: done (0 left) 2024-04-26T04:33:23.358 INFO:tasks.rados.rados.0.smithi178.stdout:385: rollback oid 39 current snap is 25 2024-04-26T04:33:23.359 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 39 to 24 2024-04-26T04:33:23.359 INFO:tasks.rados.rados.0.smithi178.stdout:386: rollback oid 21 current snap is 25 2024-04-26T04:33:23.359 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 21 to 23 2024-04-26T04:33:23.359 INFO:tasks.rados.rados.0.smithi178.stdout:387: delete oid 24 current snap is 25 2024-04-26T04:33:23.359 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:23.359 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:23.376 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T04:33:23.449 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:23.448+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:23.462 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:33:23.462 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:33:23.484 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:33:23.484 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:33:23.497 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:23.497 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:23.504 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:33:23.504 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:33:23.511 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T04:33:23.527 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:33:23.527 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:33:23.548 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:33:23.549 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:33:23.550 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:23.551+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:23.573 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:33:23.573 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:33:23.586 INFO:tasks.rados.rados.0.smithi178.stdout:386: finishing rollback tid 0 to smithi17829308-21 2024-04-26T04:33:23.598 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:33:23.598 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:33:23.620 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:33:23.620 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:33:23.635 INFO:teuthology.orchestra.run.smithi178.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:23.635 INFO:teuthology.orchestra.run.smithi178.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:23.637 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:33:23.637 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:33:23.649 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T04:33:23.651 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:33:23.646+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:33:23.662 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:33:23.662 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:33:23.685 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:33:23.685 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:33:23.687 INFO:tasks.rados.rados.0.smithi178.stdout:387: done (2 left) 2024-04-26T04:33:23.687 INFO:tasks.rados.rados.0.smithi178.stdout:388: delete oid 3 current snap is 25 2024-04-26T04:33:23.706 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:33:23.707 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:33:23.726 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:33:23.726 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:33:23.748 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:33:23.749 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:33:23.752 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:23.745+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:23.774 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:33:23.774 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:33:23.779 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:23.780 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:33:23.780 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:33:23.780 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:23.780 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.780 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:8 3.fs0 3:f50d0def:::smithi17829308-3:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:23.780 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:33:23.780 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 58.575417643999998, 2024-04-26T04:33:23.780 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51428121599999999, 2024-04-26T04:33:23.780 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.780 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.780 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.780 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.780 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 8 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.782 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.782 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200419+0000", 2024-04-26T04:33:23.782 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9800000000000002e-07 2024-04-26T04:33:23.782 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.782 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.782 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200433+0000", 2024-04-26T04:33:23.782 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4513e-05 2024-04-26T04:33:23.782 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.782 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.782 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200460+0000", 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6438999999999999e-05 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200477+0000", 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7346e-05 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200757+0000", 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000279771 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:23.783 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200766+0000", 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.7159999999999992e-06 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.341113+0000", 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14034733799999999 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.341282+0000", 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000169197 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.784 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.342226+0000", 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00094361300000000005 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.343875+0000", 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016493910000000001 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714472+0000", 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.37059684599999998 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.785 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.786 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714699+0000", 2024-04-26T04:33:23.786 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022704800000000001 2024-04-26T04:33:23.786 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.786 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.786 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.786 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:9 3.fs0 3:f50d0def:::smithi17829308-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:23.786 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:33:23.786 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 58.575336954999997, 2024-04-26T04:33:23.786 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71712659899999998, 2024-04-26T04:33:23.786 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.786 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.787 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:23.787 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.787 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.787 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.787 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 9 2024-04-26T04:33:23.787 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.787 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.787 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.787 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.787 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:33:23.787 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.787 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1999999999999998e-07 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200506+0000", 2024-04-26T04:33:23.788 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.1399999999999997e-06 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200520+0000", 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4253000000000001e-05 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200531+0000", 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1375999999999999e-05 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.789 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200817+0000", 2024-04-26T04:33:23.790 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00028585699999999998 2024-04-26T04:33:23.790 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.790 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200824+0000", 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6730000000000003e-06 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.342845+0000", 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.142020961 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.343080+0000", 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000235481 2024-04-26T04:33:23.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714851+0000", 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.371770347 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.715101+0000", 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00025083100000000002 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917625+0000", 2024-04-26T04:33:23.792 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20252376 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:16 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~98304,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 58.574179954000002, 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.514661445, 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.793 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 16 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201657+0000", 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.052e-06 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201822+0000", 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016550000000000001 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201842+0000", 2024-04-26T04:33:23.795 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0231e-05 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201856+0000", 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3234999999999999e-05 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201960+0000", 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0001043 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:23.796 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201982+0000", 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1936000000000001e-05 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344166+0000", 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14218376399999999 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344545+0000", 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00037956399999999999 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.797 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.346210+0000", 2024-04-26T04:33:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00166501 2024-04-26T04:33:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349256+0000", 2024-04-26T04:33:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0030461080000000001 2024-04-26T04:33:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716126+0000", 2024-04-26T04:33:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.366869679 2024-04-26T04:33:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.798 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716317+0000", 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019106599999999999 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:17 3.fs0 3:f03878c8:::smithi17829308-6:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 58.573951862999998, 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51482183299999995, 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.799 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 17 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.800 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:33:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201885+0000", 2024-04-26T04:33:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1060000000000001e-06 2024-04-26T04:33:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201891+0000", 2024-04-26T04:33:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.806e-06 2024-04-26T04:33:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201905+0000", 2024-04-26T04:33:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.432e-05 2024-04-26T04:33:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201929+0000", 2024-04-26T04:33:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4417e-05 2024-04-26T04:33:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202056+0000", 2024-04-26T04:33:23.802 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012684400000000001 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202066+0000", 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.5230000000000001e-06 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.347195+0000", 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.145129646 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.347408+0000", 2024-04-26T04:33:23.803 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00021322299999999999 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.348249+0000", 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000840363 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.353641+0000", 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0053921280000000004 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716520+0000", 2024-04-26T04:33:23.804 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.36287924799999999 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716705+0000", 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018520900000000001 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:18 3.fs0 3:f03878c8:::smithi17829308-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 58.573888734999997, 2024-04-26T04:33:23.805 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71589294699999995, 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 18 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:33:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201948+0000", 2024-04-26T04:33:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.11e-06 2024-04-26T04:33:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.807 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201962+0000", 2024-04-26T04:33:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4297999999999999e-05 2024-04-26T04:33:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201990+0000", 2024-04-26T04:33:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8243e-05 2024-04-26T04:33:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202004+0000", 2024-04-26T04:33:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3259e-05 2024-04-26T04:33:23.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202132+0000", 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000128596 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202149+0000", 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6463000000000001e-05 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349005+0000", 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14685641099999999 2024-04-26T04:33:23.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349233+0000", 2024-04-26T04:33:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000227798 2024-04-26T04:33:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716805+0000", 2024-04-26T04:33:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.36757169099999998 2024-04-26T04:33:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.810 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716944+0000", 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013965200000000001 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917840+0000", 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20089542599999999 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:346 3.fs0 3:fa2b8868:::smithi17829308-13:head [copy-from ver 14] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.181230+0000", 2024-04-26T04:33:23.811 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 48.594605954999999, 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.80960196600000001, 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 346 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181230+0000", 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.812 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181230+0000", 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181232+0000", 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3609999999999999e-06 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181257+0000", 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5100999999999999e-05 2024-04-26T04:33:23.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181283+0000", 2024-04-26T04:33:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5612000000000001e-05 2024-04-26T04:33:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181342+0000", 2024-04-26T04:33:23.814 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9744000000000003e-05 2024-04-26T04:33:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.817 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248343+0000", 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.067000834999999995 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248487+0000", 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014416900000000001 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.787056+0000", 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.53856828300000004 2024-04-26T04:33:23.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.789372+0000", 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0023167069999999999 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.990698+0000", 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201325688 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.990831+0000", 2024-04-26T04:33:23.819 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000133466 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:347 3.fs0 3:fa2b8868:::smithi17829308-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.181534+0000", 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 48.594301397000002, 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.60738179000000003, 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.820 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 347 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181534+0000", 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181534+0000", 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.821 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181535+0000", 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0640000000000001e-06 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181541+0000", 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4099999999999999e-06 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181557+0000", 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6728999999999999e-05 2024-04-26T04:33:23.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181570+0000", 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2743e-05 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248864+0000", 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.067293511 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248995+0000", 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013142000000000001 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.823 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.788613+0000", 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.53961845799999997 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.788760+0000", 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014664600000000001 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.788916+0000", 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000155809 2024-04-26T04:33:23.824 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:430 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~622592] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.182041+0000", 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 37.593794428000002, 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51141810899999995, 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.825 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 430 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182041+0000", 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182041+0000", 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.826 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182044+0000", 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.83e-06 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182758+0000", 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00071432400000000001 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182792+0000", 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3655999999999998e-05 2024-04-26T04:33:23.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.282928+0000", 2024-04-26T04:33:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100136005 2024-04-26T04:33:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.283058+0000", 2024-04-26T04:33:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013049700000000001 2024-04-26T04:33:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.283071+0000", 2024-04-26T04:33:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3106e-05 2024-04-26T04:33:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.326367+0000", 2024-04-26T04:33:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.043295412999999998 2024-04-26T04:33:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.326694+0000", 2024-04-26T04:33:23.829 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00032741600000000002 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.329727+0000", 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0030326509999999999 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.333822+0000", 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0040948139999999996 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.830 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.693241+0000", 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.35941913199999997 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.693459+0000", 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00021826500000000001 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:434 3.fs0 3:f03878c8:::smithi17829308-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.384120+0000", 2024-04-26T04:33:23.831 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 37.391715579, 2024-04-26T04:33:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81328199499999998, 2024-04-26T04:33:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 434 2024-04-26T04:33:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.832 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384120+0000", 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384120+0000", 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384121+0000", 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3069999999999999e-06 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384127+0000", 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4430000000000002e-06 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384147+0000", 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0533000000000001e-05 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384163+0000", 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5948999999999999e-05 2024-04-26T04:33:23.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.491986+0000", 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.107822418 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.492246+0000", 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026025100000000003 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.796841+0000", 2024-04-26T04:33:23.835 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30459499299999998 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.796996+0000", 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015495 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:47.197402+0000", 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40040615099999999 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:488 3.fs0 3:f8001b32:::smithi17829308-38:head [copy-from ver 18] snapc d=[b,a] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:51.211447+0000", 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 32.564388925000003, 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71306741799999995, 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 488 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.837 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211447+0000", 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211447+0000", 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211449+0000", 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.554e-06 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211474+0000", 2024-04-26T04:33:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4987999999999999e-05 2024-04-26T04:33:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211499+0000", 2024-04-26T04:33:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4847000000000001e-05 2024-04-26T04:33:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211516+0000", 2024-04-26T04:33:23.839 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6731e-05 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211811+0000", 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000294999 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211828+0000", 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7842999999999998e-05 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.415705+0000", 2024-04-26T04:33:23.840 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20387643899999999 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.416026+0000", 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00032070999999999998 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.819320+0000", 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40329431799999998 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.841 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.821274+0000", 2024-04-26T04:33:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001954258 2024-04-26T04:33:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.924373+0000", 2024-04-26T04:33:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103099154 2024-04-26T04:33:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.924514+0000", 2024-04-26T04:33:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014057699999999999 2024-04-26T04:33:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.842 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:489 3.fs0 3:f8001b32:::smithi17829308-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:51.211561+0000", 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 32.564274715000003, 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61242426699999997, 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 489 2024-04-26T04:33:23.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211561+0000", 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211561+0000", 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211561+0000", 2024-04-26T04:33:23.844 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1699999999999997e-07 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211566+0000", 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.2289999999999997e-06 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211576+0000", 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0543999999999999e-05 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211585+0000", 2024-04-26T04:33:23.845 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1749999999999994e-06 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211947+0000", 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00036154399999999999 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211960+0000", 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3443e-05 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.846 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.615185+0000", 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40322487499999998 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.615237+0000", 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.2206999999999997e-05 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.823589+0000", 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.208351707 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.847 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.823795+0000", 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00020569300000000001 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.823985+0000", 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019013299999999999 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:501 3.fs0 3:f8001b32:::smithi17829308-38:head [copy-from ver 2] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:52.384346+0000", 2024-04-26T04:33:23.848 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 31.391489107999998, 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.611539218, 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 501 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384346+0000", 2024-04-26T04:33:23.849 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384346+0000", 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384349+0000", 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3760000000000002e-06 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384360+0000", 2024-04-26T04:33:23.850 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1240000000000001e-05 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384436+0000", 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5871999999999995e-05 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384464+0000", 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7693000000000001e-05 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.851 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.395288+0000", 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.010824926 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.395451+0000", 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000162607 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.699338+0000", 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.303887405 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.792390+0000", 2024-04-26T04:33:23.852 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.093051153999999997 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.995723+0000", 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.203333075 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.995886+0000", 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016286999999999999 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.853 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.854 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.854 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:817 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~638976] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:23.854 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.281200+0000", 2024-04-26T04:33:23.854 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.494635864999999, 2024-04-26T04:33:23.854 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51066042700000003, 2024-04-26T04:33:23.854 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.854 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.854 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.854 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.854 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.854 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.854 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 817 2024-04-26T04:33:23.854 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.281200+0000", 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.281200+0000", 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.855 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.281202+0000", 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8949999999999998e-06 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.281629+0000", 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00042693800000000002 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.281661+0000", 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1429e-05 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.856 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.381806+0000", 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100144873 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382002+0000", 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019577700000000001 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382014+0000", 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2694000000000001e-05 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.857 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.483589+0000", 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101574648 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.483736+0000", 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014714400000000001 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.485464+0000", 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001728151 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.858 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.593922+0000", 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10845802 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.791793+0000", 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.19787133000000001 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.791860+0000", 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6527999999999997e-05 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.859 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:819 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~802816] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.382377+0000", 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.393458369000001, 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.510421554, 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 819 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.860 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382377+0000", 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382377+0000", 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382378+0000", 2024-04-26T04:33:23.861 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.75e-07 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382790+0000", 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00041200200000000002 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382799+0000", 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6749999999999997e-06 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.862 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382804+0000", 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5109999999999997e-06 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.482975+0000", 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10017138 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.482987+0000", 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.163e-05 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.863 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.490398+0000", 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0074111580000000002 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.490664+0000", 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000265596 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.492500+0000", 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001835857 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.864 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.593962+0000", 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10146279800000001 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.892700+0000", 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.298737366 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.892799+0000", 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8906000000000005e-05 2024-04-26T04:33:23.865 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:820 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~81920] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.382815+0000", 2024-04-26T04:33:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.393020032999999, 2024-04-26T04:33:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61062931799999998, 2024-04-26T04:33:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.866 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 820 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382815+0000", 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382815+0000", 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382816+0000", 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8400000000000004e-07 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382850+0000", 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3955999999999998e-05 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382857+0000", 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.0210000000000002e-06 2024-04-26T04:33:23.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382861+0000", 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.9489999999999998e-06 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.483105+0000", 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100243689 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.483116+0000", 2024-04-26T04:33:23.869 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0943e-05 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.594521+0000", 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.11140509699999999 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.594688+0000", 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016764500000000001 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.870 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.595346+0000", 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00065723599999999995 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.596772+0000", 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001426411 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.993251+0000", 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39647891000000002 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.871 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.993445+0000", 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019387699999999999 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:821 3.fs0 3:f03878c8:::smithi17829308-6:head [setxattr _header (56) in=7b] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.382870+0000", 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.392965890999999, 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61114189399999996, 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.872 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 821 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382870+0000", 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382870+0000", 2024-04-26T04:33:23.873 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382870+0000", 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.9099999999999999e-07 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382872+0000", 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.305e-06 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.874 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382876+0000", 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.8380000000000004e-06 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382879+0000", 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8459999999999999e-06 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.595741+0000", 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21286179199999999 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.875 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.596013+0000", 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00027180599999999998 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.596574+0000", 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00056180100000000001 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.596964+0000", 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00038925 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.993808+0000", 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39684441399999998 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.994011+0000", 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000203451 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:822 3.fs0 3:f03878c8:::smithi17829308-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:23.877 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.382891+0000", 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.392944077999999, 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81435076900000003, 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 822 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.878 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382891+0000", 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382891+0000", 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382892+0000", 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8299999999999998e-07 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.879 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382899+0000", 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5009999999999998e-06 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382905+0000", 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.241e-06 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382910+0000", 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0289999999999996e-06 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.596801+0000", 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.213890254 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.596847+0000", 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.6711000000000001e-05 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.994769+0000", 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39792192399999998 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.994911+0000", 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014135800000000001 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:08.197242+0000", 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20233146799999999 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.882 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:841 3.fs0 3:f0b18d84:::smithi17829308-36:head [append 0~573440] snapc 16=[16,15,14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:09.300234+0000", 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 14.475601846, 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.52213421199999999, 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 841 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.883 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.300234+0000", 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.300234+0000", 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.300235+0000", 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7770000000000001e-06 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.884 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.300876+0000", 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00064050999999999995 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.300902+0000", 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6333e-05 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.300921+0000", 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8899999999999999e-05 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.301110+0000", 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000189321 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.301278+0000", 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000167913 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.302675+0000", 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0013964769999999999 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.304424+0000", 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001749401 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.822251+0000", 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51782681100000005 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.822368+0000", 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011676900000000001 2024-04-26T04:33:23.887 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:842 3.fs0 3:f0b18d84:::smithi17829308-36:head [append 0~753664] snapc 16=[16,15,14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:09.312604+0000", 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 14.463231374999999, 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51013875799999997, 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.888 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 842 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.312604+0000", 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.312604+0000", 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.889 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.312606+0000", 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2630000000000002e-06 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.313439+0000", 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00083306799999999996 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.313473+0000", 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3572999999999999e-05 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.890 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.313496+0000", 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3019000000000001e-05 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.313642+0000", 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014585800000000001 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.313805+0000", 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016319199999999999 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.891 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.315376+0000", 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001570833 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.318746+0000", 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0033702599999999999 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.822552+0000", 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50380583899999998 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.892 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.822743+0000", 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000190853 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:844 3.fs0 3:f0b18d84:::smithi17829308-36:head [setxattr _header (57) in=7b] snapc 16=[16,15,14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:09.316573+0000", 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 14.459262183, 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50695738099999998, 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:23.893 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 844 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.316573+0000", 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:23.894 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.316573+0000", 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.316575+0000", 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.553e-06 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.316583+0000", 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.3429999999999996e-06 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:23.895 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.316604+0000", 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0806999999999999e-05 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.416764+0000", 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100160148 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.517168+0000", 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10040428699999999 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.896 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.517342+0000", 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000173294 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.518090+0000", 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00074871499999999995 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.621319+0000", 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10322877699999999 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.897 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.823390+0000", 2024-04-26T04:33:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20207059199999999 2024-04-26T04:33:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.823531+0000", 2024-04-26T04:33:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000140865 2024-04-26T04:33:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:23.898 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:23.898 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:23.899 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:33:23.901 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T04:33:23.901 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:33:23.901 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:23.896+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:23.966 INFO:tasks.rados.rados.0.smithi178.stdout:385: finishing rollback tid 0 to smithi17829308-39 2024-04-26T04:33:23.994 INFO:tasks.rados.rados.0.smithi178.stdout:388: done (2 left) 2024-04-26T04:33:23.994 INFO:tasks.rados.rados.0.smithi178.stdout:389: read oid 1 snap 24 2024-04-26T04:33:23.994 INFO:tasks.rados.rados.0.smithi178.stdout:389: expect (ObjNum 86 snap 12 seq_num 86) 2024-04-26T04:33:23.994 INFO:tasks.rados.rados.0.smithi178.stdout:390: setattr oid 2 current snap is 25 2024-04-26T04:33:23.998 INFO:tasks.rados.rados.0.smithi178.stdout:391: setattr oid 10 current snap is 25 2024-04-26T04:33:24.000 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:33:23.998+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:33:24.001 INFO:tasks.rados.rados.0.smithi178.stdout:392: delete oid 6 current snap is 25 2024-04-26T04:33:24.076 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:24.076 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:24.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.077 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDPGPush(3.1s4 88/85 [PushOp(3:8bd869b7:::smithi17829308-2:head, version: 68'80, data_included: [262144~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:8bd869b7:::smithi17829308-2:head@68'80, size: 5160960, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 2097152, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-26T04:33:24.077 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:23.995602+0000", 2024-04-26T04:33:24.077 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 0.080980574999999999, 2024-04-26T04:33:24.077 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.081030459999999999, 2024-04-26T04:33:24.077 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.077 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.077 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:24.077 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.077 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.077 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:23.995602+0000", 2024-04-26T04:33:24.078 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.078 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.078 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:23.995602+0000", 2024-04-26T04:33:24.078 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.078 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.078 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:23.995603+0000", 2024-04-26T04:33:24.078 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9589999999999998e-06 2024-04-26T04:33:24.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.079 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.079 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:23.996164+0000", 2024-04-26T04:33:24.079 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00056089599999999996 2024-04-26T04:33:24.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.079 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.079 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:23.996198+0000", 2024-04-26T04:33:24.079 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3510999999999998e-05 2024-04-26T04:33:24.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.079 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.079 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:23.996223+0000", 2024-04-26T04:33:24.079 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5109e-05 2024-04-26T04:33:24.080 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.080 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.080 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.080 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.080 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2024-04-26T04:33:24.080 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 1 2024-04-26T04:33:24.080 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:24.090 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T04:33:24.092 INFO:tasks.rados.rados.0.smithi178.stdout:385: finishing rollback tid 1 to smithi17829308-39 2024-04-26T04:33:24.092 INFO:tasks.rados.rados.0.smithi178.stdout:385: finishing rollback tid 2 to smithi17829308-39 2024-04-26T04:33:24.092 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 39 v 98 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-26T04:33:24.101 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:24.100+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:24.202 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:24.198+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:24.208 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 10 v 110 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-26T04:33:24.209 INFO:tasks.rados.rados.0.smithi178.stdout:385: done (5 left) 2024-04-26T04:33:24.209 INFO:tasks.rados.rados.0.smithi178.stdout:391: done (4 left) 2024-04-26T04:33:24.209 INFO:tasks.rados.rados.0.smithi178.stdout:392: done (3 left) 2024-04-26T04:33:24.209 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 4 current snap is 25 2024-04-26T04:33:24.209 INFO:tasks.rados.rados.0.smithi178.stdout:393: seq_num 153 ranges {2113536=163840} 2024-04-26T04:33:24.218 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:24.218 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:24.218 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:33:24.218 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:33:24.219 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:24.233 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T04:33:24.238 INFO:tasks.rados.rados.0.smithi178.stdout:393: writing smithi17829308-4 from 2113536 to 2277376 tid 1 2024-04-26T04:33:24.238 INFO:tasks.rados.rados.0.smithi178.stdout:394: snap_remove snap 20 2024-04-26T04:33:24.293 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13, 14, 15] out_osds: [2, 10] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-26T04:33:24.294 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-26T04:33:24.294 INFO:tasks.thrashosds.thrasher:inject_pause on osd.13 2024-04-26T04:33:24.294 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-26T04:33:24.294 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-26T04:33:24.294 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok config set filestore_inject_stall 3 2024-04-26T04:33:24.341 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:24.340+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:24.364 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:24.364 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:33:24.365 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:33:24.365 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:24.365 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.365 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:21 3.5s0 3:a4a2788c:::smithi17829308-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:24.365 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:33:24.365 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 59.056207983999997, 2024-04-26T04:33:24.365 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71502945500000004, 2024-04-26T04:33:24.365 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.365 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.365 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:24.365 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.365 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.365 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.366 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 21 2024-04-26T04:33:24.366 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.366 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.366 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.366 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.366 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:33:24.366 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.366 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.366 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.366 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.366 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:33:24.366 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304624+0000", 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.186e-06 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304629+0000", 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3630000000000004e-06 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.367 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304643+0000", 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4800000000000001e-05 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304654+0000", 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0665999999999999e-05 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507573+0000", 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20291931099999999 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.368 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:24.369 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507644+0000", 2024-04-26T04:33:24.369 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1131999999999996e-05 2024-04-26T04:33:24.369 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.369 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.369 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.369 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.816490+0000", 2024-04-26T04:33:24.369 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30884542700000001 2024-04-26T04:33:24.369 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.369 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.369 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.369 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.816596+0000", 2024-04-26T04:33:24.369 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000105678 2024-04-26T04:33:24.369 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.370 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.370 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.370 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.019652+0000", 2024-04-26T04:33:24.370 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20305689199999999 2024-04-26T04:33:24.370 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.370 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.370 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.370 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.370 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.370 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:25 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~147456,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:24.370 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:33:24.370 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 59.056083256000001, 2024-04-26T04:33:24.371 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61210120899999998, 2024-04-26T04:33:24.371 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.371 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.371 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.371 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.371 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.371 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.371 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 25 2024-04-26T04:33:24.371 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.371 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.371 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.371 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.371 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:33:24.372 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.372 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.372 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.372 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.372 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:33:24.372 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.372 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.372 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.372 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.373 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304749+0000", 2024-04-26T04:33:24.373 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.277e-06 2024-04-26T04:33:24.373 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.373 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.373 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.373 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304925+0000", 2024-04-26T04:33:24.373 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017555399999999999 2024-04-26T04:33:24.373 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.373 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.373 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.373 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304943+0000", 2024-04-26T04:33:24.374 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8529000000000002e-05 2024-04-26T04:33:24.374 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.374 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.374 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.374 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304954+0000", 2024-04-26T04:33:24.374 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0879999999999999e-05 2024-04-26T04:33:24.374 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.374 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.374 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.374 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608582+0000", 2024-04-26T04:33:24.375 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30362831899999998 2024-04-26T04:33:24.375 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.375 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.375 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.375 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608711+0000", 2024-04-26T04:33:24.375 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012887 2024-04-26T04:33:24.375 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.375 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.375 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.376 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609284+0000", 2024-04-26T04:33:24.376 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00057331000000000005 2024-04-26T04:33:24.376 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.376 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.376 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.376 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.710054+0000", 2024-04-26T04:33:24.376 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100769761 2024-04-26T04:33:24.376 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.376 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.376 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.376 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.916732+0000", 2024-04-26T04:33:24.376 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20667817799999999 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.916849+0000", 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000116531 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:26 3.5s0 3:afb9923e:::smithi17829308-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 59.055861454000002, 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71271566600000003, 2024-04-26T04:33:24.377 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 26 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.378 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.2399999999999996e-07 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304975+0000", 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.6460000000000001e-06 2024-04-26T04:33:24.379 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304984+0000", 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1540000000000008e-06 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304992+0000", 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5259999999999996e-06 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609850+0000", 2024-04-26T04:33:24.380 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.304857769 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609963+0000", 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011346899999999999 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.610506+0000", 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00054322400000000003 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.381 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.710058+0000", 2024-04-26T04:33:24.382 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.099551985999999995 2024-04-26T04:33:24.382 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.382 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.382 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.382 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017545+0000", 2024-04-26T04:33:24.382 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30748642599999998 2024-04-26T04:33:24.382 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.382 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.382 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.382 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017685+0000", 2024-04-26T04:33:24.382 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014054200000000001 2024-04-26T04:33:24.382 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.382 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:360 3.5s0 3:a44e1dec:::smithi17829308-26:head [rollback 2] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.340709+0000", 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 49.020122030000003, 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.54345505299999997, 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 360 2024-04-26T04:33:24.383 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.384 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.384 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.384 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.384 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340709+0000", 2024-04-26T04:33:24.384 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.384 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.384 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.384 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.384 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340709+0000", 2024-04-26T04:33:24.384 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.384 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.384 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.384 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.385 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340710+0000", 2024-04-26T04:33:24.385 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9100000000000003e-07 2024-04-26T04:33:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340714+0000", 2024-04-26T04:33:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.1760000000000003e-06 2024-04-26T04:33:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340725+0000", 2024-04-26T04:33:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1664e-05 2024-04-26T04:33:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.386 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340734+0000", 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8899999999999996e-06 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340844+0000", 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000110005 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340851+0000", 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7830000000000001e-06 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.387 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.577334+0000", 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23648275699999999 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.577565+0000", 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00023120000000000001 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579037+0000", 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00147218 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.388 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.682611+0000", 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103574176 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.884048+0000", 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201436371 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.884164+0000", 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011616 2024-04-26T04:33:24.389 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:361 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.340770+0000", 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 49.020060809999997, 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.644132284, 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.390 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.391 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 361 2024-04-26T04:33:24.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.391 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.391 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.391 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340770+0000", 2024-04-26T04:33:24.391 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.391 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.391 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.391 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340770+0000", 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340771+0000", 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8400000000000004e-07 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340774+0000", 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.349e-06 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.392 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340783+0000", 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.3470000000000005e-06 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340792+0000", 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8740000000000001e-06 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340874+0000", 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.1458000000000005e-05 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.393 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340879+0000", 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4110000000000004e-06 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579882+0000", 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23900315899999999 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.580145+0000", 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026314300000000002 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.394 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.395 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.395 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.581275+0000", 2024-04-26T04:33:24.395 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001129997 2024-04-26T04:33:24.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.395 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.395 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.395 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.682889+0000", 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101613946 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.984790+0000", 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30190081600000002 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.984902+0000", 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0001122 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:478 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~688128] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.219546+0000", 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 34.141284669999997, 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.60491518499999997, 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 478 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.397 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.398 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.398 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219546+0000", 2024-04-26T04:33:24.398 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.398 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.398 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.398 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219546+0000", 2024-04-26T04:33:24.398 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.398 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.398 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.398 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219549+0000", 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.322e-06 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219983+0000", 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00043452399999999999 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219998+0000", 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5231999999999999e-05 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.399 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.220007+0000", 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9379999999999998e-06 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.320021+0000", 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100013729 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.320111+0000", 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9907999999999994e-05 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.400 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.320914+0000", 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00080305000000000003 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427530+0000", 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.106616091 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.824390+0000", 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.396859621 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.401 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.824461+0000", 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1769999999999999e-05 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:479 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~81920] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.221012+0000", 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 34.139819318000001, 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.70366377099999999, 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.402 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 479 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221012+0000", 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.403 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221012+0000", 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221013+0000", 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4750000000000001e-06 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221057+0000", 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.4305999999999999e-05 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.404 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221069+0000", 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1783000000000001e-05 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221077+0000", 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7649999999999993e-06 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321785+0000", 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100707597 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.405 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.322104+0000", 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00031923599999999998 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.323441+0000", 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001337233 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427532+0000", 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10409099199999999 2024-04-26T04:33:24.406 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924579+0000", 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49704692 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924675+0000", 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6464000000000003e-05 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:480 3.5s0 3:afb9923e:::smithi17829308-9:head [setxattr _header (56) in=7b] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 34.139741428999997, 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.70383227299999995, 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 480 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.408 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9999999999999997e-07 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.409 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.410 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.410 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221093+0000", 2024-04-26T04:33:24.410 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.241e-06 2024-04-26T04:33:24.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.410 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.410 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221100+0000", 2024-04-26T04:33:24.410 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6080000000000001e-06 2024-04-26T04:33:24.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.410 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.410 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.410 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221106+0000", 2024-04-26T04:33:24.410 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4999999999999999e-06 2024-04-26T04:33:24.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.411 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.411 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.324423+0000", 2024-04-26T04:33:24.411 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103317706 2024-04-26T04:33:24.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.411 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.411 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.324652+0000", 2024-04-26T04:33:24.411 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022915899999999999 2024-04-26T04:33:24.411 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.411 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.411 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.325569+0000", 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000916941 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427565+0000", 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101995135 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924820+0000", 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49725566500000001 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.412 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.413 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.413 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924922+0000", 2024-04-26T04:33:24.413 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000101618 2024-04-26T04:33:24.413 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.413 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.413 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.413 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.413 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:481 3.5s0 3:afb9923e:::smithi17829308-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:24.413 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.221113+0000", 2024-04-26T04:33:24.413 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 34.139717656000002, 2024-04-26T04:33:24.413 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.80581624500000004, 2024-04-26T04:33:24.413 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 481 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221113+0000", 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.414 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221113+0000", 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221114+0000", 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.5700000000000002e-07 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221117+0000", 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.552e-06 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.415 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221125+0000", 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7710000000000006e-06 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321214+0000", 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100088539 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.326595+0000", 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0053815479999999999 2024-04-26T04:33:24.416 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.326744+0000", 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014871700000000001 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.925438+0000", 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.59869437999999997 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.925572+0000", 2024-04-26T04:33:24.417 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000133912 2024-04-26T04:33:24.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.418 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.418 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.026930+0000", 2024-04-26T04:33:24.418 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101357269 2024-04-26T04:33:24.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.418 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.418 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.418 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.418 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:728 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~475136] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:24.418 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.306711+0000", 2024-04-26T04:33:24.418 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 22.054120447999999, 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.90587526100000004, 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 728 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.306711+0000", 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.419 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.306711+0000", 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.306713+0000", 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8389999999999998e-06 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.307016+0000", 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00030270699999999999 2024-04-26T04:33:24.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.307050+0000", 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.4099e-05 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.307069+0000", 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.836e-05 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396493+0000", 2024-04-26T04:33:24.421 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.089424218999999999 2024-04-26T04:33:24.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.422 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:24.422 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396500+0000", 2024-04-26T04:33:24.422 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.6960000000000005e-06 2024-04-26T04:33:24.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.422 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.422 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.597275+0000", 2024-04-26T04:33:24.422 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20077457100000001 2024-04-26T04:33:24.422 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.422 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.426 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.426 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.597440+0000", 2024-04-26T04:33:24.426 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000165021 2024-04-26T04:33:24.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.426 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.426 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.426 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.599124+0000", 2024-04-26T04:33:24.426 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001683882 2024-04-26T04:33:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703777+0000", 2024-04-26T04:33:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.104652914 2024-04-26T04:33:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.212458+0000", 2024-04-26T04:33:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50868092200000004 2024-04-26T04:33:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.427 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.428 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.212586+0000", 2024-04-26T04:33:24.428 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012803099999999999 2024-04-26T04:33:24.428 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.428 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.428 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.428 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.428 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:729 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~655360] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:24.428 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.316292+0000", 2024-04-26T04:33:24.428 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 22.044538486, 2024-04-26T04:33:24.428 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.89658193900000005, 2024-04-26T04:33:24.428 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.428 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 729 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316292+0000", 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.429 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316292+0000", 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316295+0000", 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4490000000000002e-06 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316680+0000", 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00038542699999999998 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.430 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316716+0000", 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.5405999999999999e-05 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316736+0000", 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0505000000000001e-05 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396572+0000", 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.079835357999999995 2024-04-26T04:33:24.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396575+0000", 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.529e-06 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.600521+0000", 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20394542500000001 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.600677+0000", 2024-04-26T04:33:24.432 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015607699999999999 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.602205+0000", 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001528661 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703787+0000", 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10158194500000001 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.433 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.212771+0000", 2024-04-26T04:33:24.434 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50898380499999996 2024-04-26T04:33:24.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.434 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.434 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.212874+0000", 2024-04-26T04:33:24.434 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000103352 2024-04-26T04:33:24.434 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.434 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.434 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.434 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.434 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:730 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~622592] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:24.434 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.325217+0000", 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 22.035614444, 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.88791738899999995, 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 730 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.435 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325217+0000", 2024-04-26T04:33:24.436 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.436 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.436 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325217+0000", 2024-04-26T04:33:24.436 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.436 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.436 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325219+0000", 2024-04-26T04:33:24.436 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.283e-06 2024-04-26T04:33:24.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.436 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325555+0000", 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00033651699999999999 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325585+0000", 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9173999999999999e-05 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325601+0000", 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6551000000000001e-05 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.437 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396592+0000", 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.070990756000000002 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396595+0000", 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.061e-06 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.603348+0000", 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.206753411 2024-04-26T04:33:24.438 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.603532+0000", 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018416400000000001 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.605092+0000", 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001559337 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703790+0000", 2024-04-26T04:33:24.439 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.098698498999999995 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213032+0000", 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50924209399999998 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213134+0000", 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000101542 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.440 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.441 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.441 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:731 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~278528] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:24.441 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.329197+0000", 2024-04-26T04:33:24.441 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 22.031633496000001, 2024-04-26T04:33:24.441 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.88429409199999998, 2024-04-26T04:33:24.441 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.441 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.441 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.441 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.441 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.441 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.441 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 731 2024-04-26T04:33:24.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.442 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.442 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.442 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329197+0000", 2024-04-26T04:33:24.442 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.442 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.442 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329197+0000", 2024-04-26T04:33:24.442 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.442 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329199+0000", 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5910000000000001e-06 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329372+0000", 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017300599999999999 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329398+0000", 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6400000000000001e-05 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.443 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329413+0000", 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4548e-05 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396610+0000", 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.067197398000000005 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396613+0000", 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.033e-06 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.444 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.606931+0000", 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21031707599999999 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.607074+0000", 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000143144 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.608197+0000", 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0011229980000000001 2024-04-26T04:33:24.445 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703793+0000", 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.095596134999999999 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213368+0000", 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50957523800000004 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213492+0000", 2024-04-26T04:33:24.446 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012352500000000001 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:732 3.5s0 3:a44e1dec:::smithi17829308-26:head [setxattr _header (57) in=7b] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.329456+0000", 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 22.031374840000002, 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.8844417, 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.447 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 732 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329456+0000", 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329456+0000", 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329457+0000", 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1969999999999999e-06 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329465+0000", 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1969999999999999e-06 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329483+0000", 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8351000000000001e-05 2024-04-26T04:33:24.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329497+0000", 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4114000000000001e-05 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.407278+0000", 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.077780571000000007 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.407289+0000", 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.111e-05 2024-04-26T04:33:24.450 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.451 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.451 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.609056+0000", 2024-04-26T04:33:24.451 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201767264 2024-04-26T04:33:24.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.451 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.451 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.609198+0000", 2024-04-26T04:33:24.451 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014221700000000001 2024-04-26T04:33:24.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.451 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.451 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.451 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.609956+0000", 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00075800100000000001 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703800+0000", 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.093843736999999997 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213750+0000", 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50995032600000001 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213898+0000", 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000147615 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:733 3.5s0 3:a44e1dec:::smithi17829308-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.329515+0000", 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 22.031315943999999, 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0903875629999999, 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.453 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.454 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.454 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 733 2024-04-26T04:33:24.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.454 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.454 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.454 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329515+0000", 2024-04-26T04:33:24.454 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.454 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.454 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.454 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329515+0000", 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329516+0000", 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.2299999999999999e-07 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329520+0000", 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5399999999999997e-06 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.455 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329537+0000", 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6393999999999999e-05 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329550+0000", 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2934e-05 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.407334+0000", 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.077784331999999998 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.456 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.407342+0000", 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.9699999999999999e-06 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703462+0000", 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29612015899999999 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703734+0000", 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00027200199999999998 2024-04-26T04:33:24.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.214603+0000", 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51086900899999999 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.214847+0000", 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00024385300000000001 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.419903+0000", 2024-04-26T04:33:24.458 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.205055447 2024-04-26T04:33:24.459 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.459 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.459 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.459 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.459 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.459 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(osd.5.23:4 3.5s0 3:afb9923e:::smithi17829308-9:head [assert-version v44,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-26T04:33:24.459 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:03.630782+0000", 2024-04-26T04:33:24.459 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 20.730049448999999, 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.60668015399999997, 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630782+0000", 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630782+0000", 2024-04-26T04:33:24.460 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630785+0000", 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3270000000000001e-06 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630798+0000", 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2893e-05 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630832+0000", 2024-04-26T04:33:24.461 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3865999999999997e-05 2024-04-26T04:33:24.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.462 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.462 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.462 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630853+0000", 2024-04-26T04:33:24.462 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1437000000000002e-05 2024-04-26T04:33:24.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.462 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.462 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.462 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.833963+0000", 2024-04-26T04:33:24.462 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20310972999999999 2024-04-26T04:33:24.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.462 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.462 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.834253+0000", 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00029035900000000002 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:04.237462+0000", 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.403208542 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:813 3.5s0 3:ac36f45f:::smithi17829308-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.365605+0000", 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 16.995225571999999, 2024-04-26T04:33:24.463 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.56179527200000001, 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 813 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365605+0000", 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.464 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365605+0000", 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365606+0000", 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9499999999999998e-07 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365607+0000", 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.339e-06 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.465 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365612+0000", 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.532e-06 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365615+0000", 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.191e-06 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.366700+0000", 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0010852710000000001 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.466 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:24.467 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.366706+0000", 2024-04-26T04:33:24.467 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.2230000000000004e-06 2024-04-26T04:33:24.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.467 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.467 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.467 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580054+0000", 2024-04-26T04:33:24.467 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21334779800000001 2024-04-26T04:33:24.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.467 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.467 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:24.467 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580129+0000", 2024-04-26T04:33:24.467 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5333000000000005e-05 2024-04-26T04:33:24.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.884613+0000", 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30448329899999999 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.884873+0000", 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026001900000000001 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.927401+0000", 2024-04-26T04:33:24.468 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.042527971999999997 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:834 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~0] snapc 16=[16,15,14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:09.269198+0000", 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 15.091633377000001, 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.610278235, 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.469 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 834 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269198+0000", 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269198+0000", 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269200+0000", 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3159999999999999e-06 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269211+0000", 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1487e-05 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269235+0000", 2024-04-26T04:33:24.471 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3476999999999999e-05 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269254+0000", 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9219e-05 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.273442+0000", 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0041875920000000004 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:24.472 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.273461+0000", 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9656999999999998e-05 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.575901+0000", 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30243927799999998 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.576065+0000", 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000164864 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.473 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.576829+0000", 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00076318299999999996 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.677695+0000", 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100866385 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.879354+0000", 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201658688 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.474 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.879476+0000", 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000122089 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:954 3.5s0 3:a5af5236:::smithi17829308-45:head [setxattr _header (57) in=7b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:19.398965+0000", 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.9618664299999997, 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61122250099999997, 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.475 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 954 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398965+0000", 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398965+0000", 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398965+0000", 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.33e-07 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398970+0000", 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.4669999999999998e-06 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398980+0000", 2024-04-26T04:33:24.477 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0003000000000001e-05 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398988+0000", 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5989999999999996e-06 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.503122+0000", 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.104134428 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.478 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.503405+0000", 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00028289800000000001 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.504791+0000", 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00138569 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.605750+0000", 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10095949999999999 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.479 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:20.010040+0000", 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40428993299999999 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:20.010187+0000", 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014705 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:955 3.5s0 3:a5af5236:::smithi17829308-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:19.398998+0000", 2024-04-26T04:33:24.480 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 4.9618326939999999, 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.91489952299999999, 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 955 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398998+0000", 2024-04-26T04:33:24.481 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.482 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.482 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:24.482 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398998+0000", 2024-04-26T04:33:24.482 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:24.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.482 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.482 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:24.482 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398999+0000", 2024-04-26T04:33:24.482 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.9599999999999998e-07 2024-04-26T04:33:24.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.482 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.482 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.399002+0000", 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9440000000000001e-06 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.399012+0000", 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9259999999999995e-06 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.399022+0000", 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6330000000000008e-06 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.483 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.706787+0000", 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30776545599999999 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.707088+0000", 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00030146399999999999 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:20.010863+0000", 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30377425800000002 2024-04-26T04:33:24.484 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:20.010993+0000", 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013003400000000001 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:20.313898+0000", 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30290501199999997 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:24.485 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:24.486 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:24.487 INFO:tasks.rados.rados.0.smithi178.stdout:394: done (4 left) 2024-04-26T04:33:24.487 INFO:tasks.rados.rados.0.smithi178.stdout:395: rollback oid 47 current snap is 25 2024-04-26T04:33:24.487 INFO:tasks.rados.rados.0.smithi178.stdout:rollback oid 47 to 24 2024-04-26T04:33:24.487 INFO:tasks.rados.rados.0.smithi178.stdout:389: expect (ObjNum 86 snap 12 seq_num 86) 2024-04-26T04:33:24.488 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-04-26T04:33:24.488 INFO:teuthology.orchestra.run.smithi113.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-26T04:33:24.488 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-04-26T04:33:24.489 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T04:33:24.489 INFO:tasks.rados.rados.0.smithi178.stdout:389: done (4 left) 2024-04-26T04:33:24.490 INFO:tasks.rados.rados.0.smithi178.stdout:396: read oid 9 snap 24 2024-04-26T04:33:24.490 INFO:tasks.rados.rados.0.smithi178.stdout:396: expect (ObjNum 11 snap 0 seq_num 11) 2024-04-26T04:33:24.490 INFO:tasks.rados.rados.0.smithi178.stdout:append oid 27 current snap is 25 2024-04-26T04:33:24.490 INFO:tasks.rados.rados.0.smithi178.stdout:397: seq_num 154 ranges {2686976=180224} 2024-04-26T04:33:24.490 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:24.487+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:24.507 INFO:tasks.rados.rados.0.smithi178.stdout:397: writing smithi17829308-27 from 2686976 to 2867200 tid 1 2024-04-26T04:33:24.507 INFO:tasks.rados.rados.0.smithi178.stdout:398: delete oid 41 current snap is 25 2024-04-26T04:33:24.507 INFO:tasks.rados.rados.0.smithi178.stdout:395: finishing rollback tid 1 to smithi17829308-47 2024-04-26T04:33:24.507 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 47 v 0 (ObjNum 161 snap 0 seq_num 1) dirty dne 2024-04-26T04:33:24.517 INFO:tasks.rados.rados.0.smithi178.stdout:393: finishing write tid 1 to smithi17829308-4 2024-04-26T04:33:24.517 INFO:tasks.rados.rados.0.smithi178.stdout:393: finishing write tid 2 to smithi17829308-4 2024-04-26T04:33:24.588 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:24.587+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:24.635 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:33:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.5s5 89/30 ECSubWrite(tid=162, reqid=osd.15.0:162, at_version=89'131, trim_to=0'0, roll_forward_to=88'129))", 2024-04-26T04:33:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:24.583893+0000", 2024-04-26T04:33:24.635 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.048248261000000001, 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.048289156999999999, 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.583893+0000", 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.583893+0000", 2024-04-26T04:33:24.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.583898+0000", 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.66e-06 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.583926+0000", 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.7424999999999999e-05 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.583943+0000", 2024-04-26T04:33:24.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7292999999999999e-05 2024-04-26T04:33:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.583957+0000", 2024-04-26T04:33:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3332999999999999e-05 2024-04-26T04:33:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:33:24.638 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 1 2024-04-26T04:33:24.638 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:24.647 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-26T04:33:24.689 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:24.687+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:24.778 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:33:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-04-26T04:33:24.778 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-04-26T04:33:24.778 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:24.790 INFO:tasks.ceph.osd.12.smithi012.stderr:2024-04-26T04:33:24.785+0000 7f93e7767640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25084) UID: 0 2024-04-26T04:33:24.793 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-26T04:33:24.813 INFO:tasks.rados.rados.0.smithi178.stdout:397: finishing write tid 1 to smithi17829308-27 2024-04-26T04:33:24.813 INFO:tasks.rados.rados.0.smithi178.stdout:397: finishing write tid 2 to smithi17829308-27 2024-04-26T04:33:24.891 INFO:tasks.ceph.osd.6.smithi137.stderr:2024-04-26T04:33:24.891+0000 7fc44f2fc640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25089) UID: 0 2024-04-26T04:33:24.906 INFO:tasks.rados.rados.0.smithi178.stdout:395: done (6 left) 2024-04-26T04:33:24.906 INFO:tasks.rados.rados.0.smithi178.stdout:398: done (5 left) 2024-04-26T04:33:24.906 INFO:tasks.rados.rados.0.smithi178.stdout:399: read oid 32 snap 24 2024-04-26T04:33:24.906 INFO:tasks.rados.rados.0.smithi178.stdout:399: expect (ObjNum 142 snap 23 seq_num 142) 2024-04-26T04:33:24.906 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 6 2024-04-26T04:33:24.914 INFO:tasks.rados.rados.0.smithi178.stdout:396: expect (ObjNum 11 snap 0 seq_num 11) 2024-04-26T04:33:24.927 INFO:tasks.rados.rados.0.smithi178.stdout:396: done (5 left) 2024-04-26T04:33:24.927 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 5 2024-04-26T04:33:24.972 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:24.973 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-04-26T04:33:24.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-04-26T04:33:24.973 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:33:24.973 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.973 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:320 3.6s0 3:647b920d:::smithi17829308-33:head [create,setxattr eROReghfPdlf^m`e`^hhNZXOkdsQ]]kX`senUiSZ (875) in=40b,setxattr s`rNbaqcP\\diTaoZYXg[m[inZ`]QtZdjom`_dhUX (1928) in=40b,setxattr [[YXf (69) in=5b,setxattr ]rjrtd]YkcY\\lQ[bOp_lNWs\\NOirOQ[Ri`rndifj (5260) in=40b,setxattr n[ZSX[Rc`\\dlYpV^rPdsQfnbg^l]ioSo^`oZsXaf (19016) in=40b,setxattr b\\ffPRnhZeR\\shWQYPRaZVroNXYoUYf[eemPeQen (1648) in=40b,setxattr [gNas\\_VhjX_YRX[aQbekgbWYg`P_SWgskeiZbkP (1333) in=40b,setxattr Ysi^k[[WWsYagaeecamn^bcecbfS]p^i^jfgsmTY (6736) in=40b,setxattr Nm_N]c]XYfmpWd\\aiW^bPSdY]TjNrnirngZdd_jb (103) in=40b,setxattr cisVjO^iia\\[tQaXjaroW`VRNl[aiYR\\tcnhVsfb (8841) in=40b,setxattr Zgg]\\aOQfmaVkY (69) in=14b,setxattr X`i_`tsrZ[XRfqacNap`[O`nSf]O`[Z[Oc[aenTc (1647) in=40b,setxattr oV[OofncW[WdO[XrprmdpkoptWZRfcPm^Rkernnf (34) in=40b,setxattr \\UQYgqaahhcRrkP`h`SgamahYafPmsojP_snXkmc (314) in=40b,setxattr pjilgXb]Qkl]gt__a_ZrtRptOPONOVrsmqcZ__^T (840) in=40b,setxattr Wa`ZaT[b]kbqdZrfSUkVafjqXod\\qd (69) in=30b,setxattr [YaXgtehetrqtspbYYY[ZqritVsbZeW[ak`ma``l (418) in=40b,setxattr hPanNtTRrfi`tYcdO]eRl]`fj`hllRNnXThl\\YO (69) in=39b,setxattr S_`Ri[Qj]fZqXkqqZYcte[N]Wq\\cQ]PlQdllcb]s (1542) in=40b,setxattr ttkZc`XVXmWiqdYWeagrmr^i[OUs^jpafRbOUWdp (594) in=40b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:24.973 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:34.392445+0000", 2024-04-26T04:33:24.973 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.576249971000003, 2024-04-26T04:33:24.973 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.331045494, 2024-04-26T04:33:24.973 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:24.973 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:24.973 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 320 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392445+0000", 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:24.974 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392445+0000", 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392450+0000", 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6569999999999996e-06 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392623+0000", 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017330500000000001 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.975 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.392664+0000", 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0399999999999999e-05 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.492771+0000", 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10010693 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.520054+0000", 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.027283827 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.976 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:24.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.520231+0000", 2024-04-26T04:33:24.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017687399999999999 2024-04-26T04:33:24.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.521930+0000", 2024-04-26T04:33:24.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0016988629999999999 2024-04-26T04:33:24.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.977 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.977 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.977 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.722167+0000", 2024-04-26T04:33:24.977 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20023658899999999 2024-04-26T04:33:24.977 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.722468+0000", 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00030164599999999998 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:34.723491+0000", 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010224030000000001 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.978 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:340 3.6s0 3:674cc614:::smithi17829308-50:head [append 0~573440 in=573440b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.177236+0000", 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 49.791458814999999, 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50180553100000003, 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 340 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.979 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:24.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.177236+0000", 2024-04-26T04:33:24.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:24.980 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.980 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.980 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:24.980 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.177236+0000", 2024-04-26T04:33:24.980 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:24.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.981 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:24.981 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.177239+0000", 2024-04-26T04:33:24.981 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5100000000000001e-06 2024-04-26T04:33:24.981 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.981 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178007+0000", 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00076857200000000003 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178041+0000", 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3909999999999999e-05 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178057+0000", 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5906999999999999e-05 2024-04-26T04:33:24.982 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178187+0000", 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012976099999999999 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.178351+0000", 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016396800000000001 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.179822+0000", 2024-04-26T04:33:24.983 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001470507 2024-04-26T04:33:24.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.185617+0000", 2024-04-26T04:33:24.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0057954089999999996 2024-04-26T04:33:24.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:24.984 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.678895+0000", 2024-04-26T04:33:24.984 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49327816899999999 2024-04-26T04:33:24.984 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.984 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.984 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679042+0000", 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000146818 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:341 3.6s0 3:674cc614:::smithi17829308-50:head [append 0~196608 in=196608b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180077+0000", 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 49.788617750999997, 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49926801199999998, 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:24.985 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.986 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:24.986 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:24.986 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.986 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 341 2024-04-26T04:33:24.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.986 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:24.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:24.986 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180077+0000", 2024-04-26T04:33:24.986 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:24.986 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.986 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.986 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180077+0000", 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180081+0000", 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2459999999999999e-06 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180497+0000", 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00041676000000000001 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.987 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:24.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180529+0000", 2024-04-26T04:33:24.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1497e-05 2024-04-26T04:33:24.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.988 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.988 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.988 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180556+0000", 2024-04-26T04:33:24.988 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6594e-05 2024-04-26T04:33:24.988 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.181275+0000", 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00071977499999999997 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.181487+0000", 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000211855 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.182884+0000", 2024-04-26T04:33:24.989 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00139722 2024-04-26T04:33:24.990 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.990 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.990 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.990 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287222+0000", 2024-04-26T04:33:24.990 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10433723 2024-04-26T04:33:24.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:24.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679247+0000", 2024-04-26T04:33:24.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.392025711 2024-04-26T04:33:24.991 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.991 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.991 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:24.991 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679345+0000", 2024-04-26T04:33:24.991 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8123999999999998e-05 2024-04-26T04:33:24.991 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:24.991 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:24.991 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:24.992 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.992 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.992 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:342 3.6s0 3:674cc614:::smithi17829308-50:head [setxattr _header (56) in=63b] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:24.992 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180608+0000", 2024-04-26T04:33:24.992 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 49.788087117000003, 2024-04-26T04:33:24.992 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49908019100000001, 2024-04-26T04:33:24.992 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:24.992 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:24.992 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:24.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:24.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:24.992 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.992 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 342 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180608+0000", 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180608+0000", 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.993 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180610+0000", 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8029999999999999e-06 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180619+0000", 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.3170000000000008e-06 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180642+0000", 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2442999999999998e-05 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.994 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180658+0000", 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6779999999999999e-05 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.183978+0000", 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.003319598 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.184127+0000", 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014915100000000001 2024-04-26T04:33:24.995 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.185029+0000", 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00090211999999999996 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287425+0000", 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10239564600000001 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679540+0000", 2024-04-26T04:33:24.996 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39211458199999999 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.679688+0000", 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000148751 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:343 3.6s0 3:674cc614:::smithi17829308-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180684+0000", 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 49.788011230000002, 2024-04-26T04:33:24.997 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.71138035399999999, 2024-04-26T04:33:24.998 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:24.998 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:24.998 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:24.998 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:24.998 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:24.998 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:24.998 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 343 2024-04-26T04:33:24.998 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.998 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:24.998 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.998 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:24.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180684+0000", 2024-04-26T04:33:24.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:24.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:24.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180684+0000", 2024-04-26T04:33:24.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:24.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:24.999 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:24.999 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:24.999 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180685+0000", 2024-04-26T04:33:24.999 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2869999999999999e-06 2024-04-26T04:33:24.999 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180691+0000", 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0929999999999999e-06 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180712+0000", 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0514e-05 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180728+0000", 2024-04-26T04:33:25.000 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5665999999999999e-05 2024-04-26T04:33:25.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.001 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.001 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.001 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.286928+0000", 2024-04-26T04:33:25.001 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.106200577 2024-04-26T04:33:25.001 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:25.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.287076+0000", 2024-04-26T04:33:25.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014773100000000001 2024-04-26T04:33:25.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.680466+0000", 2024-04-26T04:33:25.002 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39339001800000001 2024-04-26T04:33:25.002 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.002 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.002 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.002 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.680567+0000", 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100756 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.892064+0000", 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.211497712 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:440 3.6s0 3:674cc614:::smithi17829308-50:head [create,setxattr pnc]aaY (11788) in=7b,setxattr ZTVVStUpk__YaS\\blR\\XgesYT[Or[[aUett_nil] (1859) in=40b,setxattr ZtotWOVcPP_oiTUUTlqtYZsNneQnmhc\\bNdNpsgo (1402) in=40b,setxattr cR\\akUitgYfao[oisZ^ach\\i_kdge\\[c_mbmUf]^ (313) in=40b,setxattr pRe[UcWhXR^oagaSYScYPQa]r[hdldO\\S`V^[nh^ (874) in=40b,setxattr srVXOoW^mbi\\ccUpc`barbsc (11788) in=24b,setxattr pqjhUkafUjRqdpObgi_ZX^QV (11788) in=24b,setxattr q]Of\\bd_VgdoaP\\k^tmmpUqq^otrVVOdrtgU]nVk (12280) in=40b,setxattr ]Ple__\\nrXPQYecmQXf_o^bgN`mofssSPQrYslSO (1859) in=40b,setxattr ZiQcjT`UZQ]es__dmo^h]NdV^`qT_hniQtVY`WsT (770) in=40b,setxattr gpT^oWmeYZpbVdojjWnUWO[YfZptiQhdWdhs]Zo^ (841) in=40b,setxattr `Zgeqf\\Rb][aUdTVXYhj[ealjcUnRaiY (11788) in=32b,setxattr [dN]aooYVPYS^Q`_`bqSNVSS^`ZT\\csYZSrnjtUT (11296) in=40b,setxattr ehcNktRfRjqWWi[^XWcZokN]Wb[YnkeNl\\lkt_t\\ (1262) in=40b,setxattr OdsZcnit[de]qNV (11788) in=15b,setxattr ^]^lenZo\\TjoZOpONRno]jtaWOd`tqNdWcYccWNk (560) in=40b,setxattr ZUohXUqoPmYaaTceWo\\SRigth\\qP[Tg_^^asjknX (1085) in=40b,setxattr lR`dUarlc[n`ctj^TlY\\QpagN_doPfV`lZQ^kNtY (526) in=40b,setxattr [iXNTskqfmN\\qVhqrm`rdXpaN_VaeRbd^O\\Z_gcY (1261) in=40b,setxattr t_jR_akNV_aXVR^icaga]cP]baNne_f_UdsoeVUk (1894) in=40b] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:46.229654+0000", 2024-04-26T04:33:25.003 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 38.739040954000004, 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.51173439300000001, 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 440 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.229654+0000", 2024-04-26T04:33:25.004 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.229654+0000", 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.229658+0000", 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0999999999999997e-06 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.005 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.229850+0000", 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000191838 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.229897+0000", 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6409999999999998e-05 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.330001+0000", 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100104415 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.006 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.330173+0000", 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017236399999999999 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.330497+0000", 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032375699999999998 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.331849+0000", 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001352294 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.007 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.336320+0000", 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0044704990000000002 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.741060+0000", 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.40474016499999999 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.741389+0000", 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032855100000000001 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.008 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.009 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.009 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.009 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.009 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:640 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~638976] snapc f=[f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:33:25.009 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:57.184457+0000", 2024-04-26T04:33:25.009 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.784238245000001, 2024-04-26T04:33:25.009 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.51936972400000003, 2024-04-26T04:33:25.009 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.009 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.009 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.009 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.009 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.009 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 640 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.184457+0000", 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.184457+0000", 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.010 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.184460+0000", 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8899999999999999e-06 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.185893+0000", 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001433141 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.185929+0000", 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6316999999999998e-05 2024-04-26T04:33:25.011 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.185946+0000", 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7085999999999999e-05 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286217+0000", 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10027051400000001 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op must wait for map", 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286230+0000", 2024-04-26T04:33:25.012 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2913e-05 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.390036+0000", 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.103806092 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.390304+0000", 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026764699999999998 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.013 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.393740+0000", 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0034366779999999999 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.602743+0000", 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20900233100000001 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.703687+0000", 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100944345 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.014 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.703827+0000", 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013977 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:641 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~458752] snapc f=[f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:57.188152+0000", 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.78054307, 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.516167932, 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.015 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 641 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.188152+0000", 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.188152+0000", 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.188154+0000", 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2239999999999998e-06 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.188810+0000", 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00065584599999999997 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.188843+0000", 2024-04-26T04:33:25.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2437000000000001e-05 2024-04-26T04:33:25.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.188858+0000", 2024-04-26T04:33:25.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.571e-05 2024-04-26T04:33:25.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.018 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.018 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286333+0000", 2024-04-26T04:33:25.018 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.097474855999999999 2024-04-26T04:33:25.018 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.018 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286345+0000", 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1353e-05 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.395969+0000", 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.109624634 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.396150+0000", 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000180674 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.019 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.397825+0000", 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001675097 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.602815+0000", 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.204989807 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.704208+0000", 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101393371 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.020 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.704320+0000", 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00011192300000000001 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:642 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~704512] snapc f=[f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:57.193975+0000", 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.774719868999998, 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.51105824799999999, 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.021 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.022 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.022 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.022 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.022 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.022 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 642 2024-04-26T04:33:25.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.022 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.022 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.022 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.193975+0000", 2024-04-26T04:33:25.022 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.022 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.022 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.193975+0000", 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.193978+0000", 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1569999999999998e-06 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.194951+0000", 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00097345999999999999 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.023 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.194983+0000", 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1603000000000002e-05 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.194998+0000", 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5586999999999998e-05 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286444+0000", 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.091445968000000002 2024-04-26T04:33:25.024 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286455+0000", 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1250000000000001e-05 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.499305+0000", 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.21284921200000001 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.499480+0000", 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000175434 2024-04-26T04:33:25.025 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.501311+0000", 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018308199999999999 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.602818+0000", 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101507166 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.704928+0000", 2024-04-26T04:33:25.026 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.102110311 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.705034+0000", 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010527999999999999 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:643 3.6s0 3:647b920d:::smithi17829308-33:head [setxattr _header (57) in=7b] snapc f=[f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:57.195028+0000", 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.773667684999999, 2024-04-26T04:33:25.027 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.81557969699999999, 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 643 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195028+0000", 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.028 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195028+0000", 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195029+0000", 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.06e-06 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195034+0000", 2024-04-26T04:33:25.029 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6829999999999996e-06 2024-04-26T04:33:25.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195050+0000", 2024-04-26T04:33:25.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6056999999999999e-05 2024-04-26T04:33:25.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.030 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195059+0000", 2024-04-26T04:33:25.030 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0499999999999997e-06 2024-04-26T04:33:25.030 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.030 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.030 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286572+0000", 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.091513091000000005 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286596+0000", 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3617000000000001e-05 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.705883+0000", 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.41928652700000002 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.031 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.706176+0000", 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029326599999999998 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.707464+0000", 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001287727 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.809419+0000", 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101955776 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.032 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:58.010340+0000", 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20092080400000001 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:58.010607+0000", 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000267039 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.033 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:644 3.6s0 3:647b920d:::smithi17829308-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:57.195074+0000", 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 27.773621452, 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.217959426, 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 644 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.034 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195074+0000", 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195074+0000", 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195075+0000", 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3300000000000001e-07 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.035 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195078+0000", 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.6669999999999998e-06 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195093+0000", 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4413000000000001e-05 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.195101+0000", 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6000000000000007e-06 2024-04-26T04:33:25.036 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286649+0000", 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.091547484999999998 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.286657+0000", 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7900000000000005e-06 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.708272+0000", 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.42161408700000003 2024-04-26T04:33:25.037 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:57.708548+0000", 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027624000000000003 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:58.011546+0000", 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.30299795800000001 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:58.011801+0000", 2024-04-26T04:33:25.038 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00025481000000000001 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:58.413033+0000", 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.401232643 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:749 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~524288] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.399186+0000", 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.569509289999999, 2024-04-26T04:33:25.039 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50750576999999997, 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 749 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.399186+0000", 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.040 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.399186+0000", 2024-04-26T04:33:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.399190+0000", 2024-04-26T04:33:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0500000000000002e-06 2024-04-26T04:33:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.041 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.400238+0000", 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00104769 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.400274+0000", 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5830000000000001e-05 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.400298+0000", 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.474e-05 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.042 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.400479+0000", 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018065999999999999 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.401431+0000", 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00095188300000000005 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404962+0000", 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0035312270000000001 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.043 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.416625+0000", 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.011662522 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.906553+0000", 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.48992856499999998 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.906692+0000", 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000138603 2024-04-26T04:33:25.044 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:750 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~573440] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.403748+0000", 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.564946812999999, 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50340580899999998, 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.045 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 750 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.403748+0000", 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.403748+0000", 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.046 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.403752+0000", 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4240000000000002e-06 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404702+0000", 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00095022599999999998 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404728+0000", 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5530000000000001e-05 2024-04-26T04:33:25.047 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.048 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.049 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.049 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404742+0000", 2024-04-26T04:33:25.049 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4867e-05 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.408057+0000", 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0033145869999999999 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.408241+0000", 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000184303 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.050 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.409896+0000", 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0016550359999999999 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.416851+0000", 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0069544619999999998 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907022+0000", 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49017117799999999 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.051 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907154+0000", 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00013219600000000001 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:751 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~688128] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.408886+0000", 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.559808902, 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49855173800000002, 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.052 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 751 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.408886+0000", 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.053 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.408886+0000", 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.408889+0000", 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5830000000000001e-06 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.410008+0000", 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001119292 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.054 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.410038+0000", 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9714e-05 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.410056+0000", 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8490000000000001e-05 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411114+0000", 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001057842 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.055 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411263+0000", 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014863699999999999 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.413187+0000", 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0019243719999999999 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.416854+0000", 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0036668899999999999 2024-04-26T04:33:25.056 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907328+0000", 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.490473822 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907438+0000", 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000110096 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.057 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.058 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:752 3.6s0 3:647b920d:::smithi17829308-33:head [append 0~294912] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:25.058 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.410845+0000", 2024-04-26T04:33:25.058 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.557850151, 2024-04-26T04:33:25.058 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49685625700000002, 2024-04-26T04:33:25.058 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.058 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.058 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.058 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.058 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.058 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.058 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 752 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.410845+0000", 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.410845+0000", 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.059 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.060 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.410847+0000", 2024-04-26T04:33:25.060 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.8870000000000001e-06 2024-04-26T04:33:25.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411302+0000", 2024-04-26T04:33:25.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00045527600000000001 2024-04-26T04:33:25.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.061 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411327+0000", 2024-04-26T04:33:25.061 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4600000000000002e-05 2024-04-26T04:33:25.061 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.061 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.061 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411341+0000", 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4100000000000001e-05 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.414402+0000", 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0030614750000000001 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.414568+0000", 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016537999999999999 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.062 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.415660+0000", 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010918620000000001 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.418568+0000", 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0029087280000000002 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907598+0000", 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.48902954599999998 2024-04-26T04:33:25.063 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907701+0000", 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000103403 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:753 3.6s0 3:647b920d:::smithi17829308-33:head [setxattr _header (57) in=7b] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.411373+0000", 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.557322314, 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.496728699, 2024-04-26T04:33:25.064 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 753 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411373+0000", 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.065 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411373+0000", 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411374+0000", 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1629999999999999e-06 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411379+0000", 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.1370000000000001e-06 2024-04-26T04:33:25.066 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411396+0000", 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6330000000000001e-05 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411405+0000", 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0599999999999997e-06 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.416890+0000", 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.005485696 2024-04-26T04:33:25.067 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.068 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.068 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.068 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.417027+0000", 2024-04-26T04:33:25.068 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000137026 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.417828+0000", 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00080056599999999995 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.419274+0000", 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001446396 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.069 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.907952+0000", 2024-04-26T04:33:25.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.48867820200000001 2024-04-26T04:33:25.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.070 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.070 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.908102+0000", 2024-04-26T04:33:25.070 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000149123 2024-04-26T04:33:25.070 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.070 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.070 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.070 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.070 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:754 3.6s0 3:647b920d:::smithi17829308-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.411418+0000", 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 22.557276758, 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.60016049599999999, 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 754 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.071 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411418+0000", 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411418+0000", 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411419+0000", 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4000000000000001e-07 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.072 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411423+0000", 2024-04-26T04:33:25.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.4400000000000001e-06 2024-04-26T04:33:25.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411434+0000", 2024-04-26T04:33:25.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1707e-05 2024-04-26T04:33:25.073 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.073 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.073 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.073 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.411443+0000", 2024-04-26T04:33:25.073 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9129999999999992e-06 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.418404+0000", 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0069603850000000004 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.418547+0000", 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000143537 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.074 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.908913+0000", 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49036608399999998 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.909197+0000", 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00028418999999999997 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.011579+0000", 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.1023815 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.075 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 68/30 ECSubWriteReply(tid=66, last_complete=68'47, committed=1, applied=1))", 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:03.393865+0000", 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.574830345999999, 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.42230022499999997, 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.393865+0000", 2024-04-26T04:33:25.076 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.393865+0000", 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.393867+0000", 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.714e-06 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.393876+0000", 2024-04-26T04:33:25.077 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6059999999999992e-06 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.393905+0000", 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9244e-05 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.494075+0000", 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10016963800000001 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.078 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.816132+0000", 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.32205738 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.816165+0000", 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2642999999999999e-05 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 68/30 ECSubWriteReply(tid=66, last_complete=68'47, committed=1, applied=1))", 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:03.518321+0000", 2024-04-26T04:33:25.079 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.450373935999998, 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39861305200000002, 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518321+0000", 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518321+0000", 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.080 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518323+0000", 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.393e-06 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518332+0000", 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6039999999999999e-06 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518368+0000", 2024-04-26T04:33:25.081 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5269999999999999e-05 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518388+0000", 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.031e-05 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.916728+0000", 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39833988199999998 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.082 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.916934+0000", 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00020659300000000001 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 68/30 ECSubWriteReply(tid=67, last_complete=68'49, committed=1, applied=1))", 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:03.518571+0000", 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 21.450124549000002, 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.59897118599999999, 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:33:25.083 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518571+0000", 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518571+0000", 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518572+0000", 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1429999999999999e-06 2024-04-26T04:33:25.084 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518579+0000", 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6529999999999994e-06 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518601+0000", 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1474000000000001e-05 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.518613+0000", 2024-04-26T04:33:25.085 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1600000000000001e-05 2024-04-26T04:33:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:04.116715+0000", 2024-04-26T04:33:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.59810200499999999 2024-04-26T04:33:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:04.117542+0000", 2024-04-26T04:33:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00082731099999999995 2024-04-26T04:33:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.086 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.087 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.087 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.087 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:25.088 INFO:tasks.rados.rados.0.smithi178.stdout:397: finishing write tid 3 to smithi17829308-27 2024-04-26T04:33:25.088 INFO:tasks.rados.rados.0.smithi178.stdout:397: oid 27 updating version 0 to 66 2024-04-26T04:33:25.088 INFO:tasks.rados.rados.0.smithi178.stdout:397: oid 27 version 66 is already newer than 65 2024-04-26T04:33:25.088 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 27 v 66 (ObjNum 154 snap 25 seq_num 154) dirty exists 2024-04-26T04:33:25.088 INFO:tasks.rados.rados.0.smithi178.stdout:397: left oid 27 (ObjNum 154 snap 25 seq_num 154) 2024-04-26T04:33:25.089 INFO:tasks.rados.rados.0.smithi178.stdout:397: done (4 left) 2024-04-26T04:33:25.089 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 4 2024-04-26T04:33:25.089 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-26T04:33:25.090 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:25.084+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:25.112 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 2 v 103 (ObjNum 88 snap 12 seq_num 88) dirty exists 2024-04-26T04:33:25.112 INFO:tasks.rados.rados.0.smithi178.stdout:390: done (3 left) 2024-04-26T04:33:25.112 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 3 2024-04-26T04:33:25.189 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:25.187+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:25.225 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:25.226 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:33:25.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.226 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpWrite(3.5s2 89/30 ECSubWrite(tid=166, reqid=osd.15.0:165, at_version=0'0, trim_to=0'0, roll_forward_to=89'133))", 2024-04-26T04:33:25.226 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:25.209754+0000", 2024-04-26T04:33:25.226 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.015553914, 2024-04-26T04:33:25.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.015593660000000001, 2024-04-26T04:33:25.226 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.226 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.226 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "no flag points reached", 2024-04-26T04:33:25.226 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.209754+0000", 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.209754+0000", 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.209755+0000", 2024-04-26T04:33:25.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6199999999999999e-06 2024-04-26T04:33:25.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.209777+0000", 2024-04-26T04:33:25.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1212999999999998e-05 2024-04-26T04:33:25.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.209809+0000", 2024-04-26T04:33:25.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1977999999999998e-05 2024-04-26T04:33:25.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.228 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpReadReply(3.1s0 89/85 ECSubReadReply(tid=416, attrs_read=0))", 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:25.220365+0000", 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 0.0049425129999999999, 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0050592700000000003, 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "queued for pg", 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.220365+0000", 2024-04-26T04:33:25.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.220365+0000", 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.220366+0000", 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1769999999999999e-06 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.220743+0000", 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00037667899999999999 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.220763+0000", 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0049999999999999e-05 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.220778+0000", 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.539e-05 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.231 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.232 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.232 INFO:teuthology.orchestra.run.smithi012.stdout: ], 2024-04-26T04:33:25.232 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 2 2024-04-26T04:33:25.232 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:25.241 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-26T04:33:25.242 INFO:tasks.rados.rados.0.smithi178.stdout:399: expect (ObjNum 142 snap 23 seq_num 142) 2024-04-26T04:33:25.290 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:25.288+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:25.374 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:25.374 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:33:25.375 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-04-26T04:33:25.375 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-04-26T04:33:25.375 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:25.383 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-26T04:33:25.393 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:25.391+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:25.394 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-26T04:33:25.492 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:25.491+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:25.495 INFO:tasks.rados.rados.0.smithi178.stdout:393: finishing write tid 3 to smithi17829308-4 2024-04-26T04:33:25.495 INFO:tasks.rados.rados.0.smithi178.stdout:393: oid 4 updating version 0 to 113 2024-04-26T04:33:25.495 INFO:tasks.rados.rados.0.smithi178.stdout:393: oid 4 version 113 is already newer than 112 2024-04-26T04:33:25.495 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 4 v 113 (ObjNum 153 snap 25 seq_num 153) dirty exists 2024-04-26T04:33:25.495 INFO:tasks.rados.rados.0.smithi178.stdout:393: left oid 4 (ObjNum 153 snap 25 seq_num 153) 2024-04-26T04:33:25.495 INFO:tasks.rados.rados.0.smithi178.stdout:393: done (2 left) 2024-04-26T04:33:25.495 INFO:tasks.rados.rados.0.smithi178.stdout:399: done (1 left) 2024-04-26T04:33:25.495 INFO:tasks.rados.rados.0.smithi178.stdout: waiting on 1 2024-04-26T04:33:25.563 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:25.563 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-04-26T04:33:25.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-04-26T04:33:25.564 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:33:25.564 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.564 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:236 3.1s0 3:8f617591:::smithi17829308-17:head [delete] snapc 2=[2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e35)", 2024-04-26T04:33:25.564 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:28.808956+0000", 2024-04-26T04:33:25.564 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 56.752721782000002, 2024-04-26T04:33:25.564 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.92629017999999996, 2024-04-26T04:33:25.564 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.564 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.564 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.564 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 236 2024-04-26T04:33:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.808956+0000", 2024-04-26T04:33:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.565 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.808956+0000", 2024-04-26T04:33:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.808958+0000", 2024-04-26T04:33:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9800000000000001e-06 2024-04-26T04:33:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.808967+0000", 2024-04-26T04:33:25.566 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8230000000000006e-06 2024-04-26T04:33:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.808987+0000", 2024-04-26T04:33:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9664999999999999e-05 2024-04-26T04:33:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:28.808998+0000", 2024-04-26T04:33:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0740999999999999e-05 2024-04-26T04:33:25.567 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.568 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.568 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.568 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.630849+0000", 2024-04-26T04:33:25.568 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.821851424 2024-04-26T04:33:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.631005+0000", 2024-04-26T04:33:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000155818 2024-04-26T04:33:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.632065+0000", 2024-04-26T04:33:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0010597359999999999 2024-04-26T04:33:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.569 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.633919+0000", 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0018537479999999999 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.735099+0000", 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101180744 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.735246+0000", 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000146501 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.570 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:240 3.1s0 3:83946740:::smithi17829308-29:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:29.766578+0000", 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 55.795099413999999, 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.795088193, 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 240 2024-04-26T04:33:25.571 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.766578+0000", 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.766578+0000", 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.572 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.766579+0000", 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6300000000000004e-07 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.766582+0000", 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3699999999999999e-06 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.766612+0000", 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9963000000000001e-05 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.573 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.766624+0000", 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1557e-05 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.048520+0000", 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.28189626099999998 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.048753+0000", 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000232699 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.574 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.267779+0000", 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.219026107 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.268049+0000", 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027038700000000003 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.561666+0000", 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.29361708600000003 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.575 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:251 3.1s0 3:8f617591:::smithi17829308-17:head [copy-from ver 5] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:29.790926+0000", 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 55.770751664000002, 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.97147092499999999, 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.576 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 251 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.790926+0000", 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.790926+0000", 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.577 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.790928+0000", 2024-04-26T04:33:25.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2139999999999999e-06 2024-04-26T04:33:25.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.578 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.790940+0000", 2024-04-26T04:33:25.578 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1542999999999999e-05 2024-04-26T04:33:25.578 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.578 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.578 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.790970+0000", 2024-04-26T04:33:25.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0574000000000002e-05 2024-04-26T04:33:25.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.790985+0000", 2024-04-26T04:33:25.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5299999999999999e-05 2024-04-26T04:33:25.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.579 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.579 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.579 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.048921+0000", 2024-04-26T04:33:25.579 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.25793591700000001 2024-04-26T04:33:25.579 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.049220+0000", 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00029858 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.375684+0000", 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.32646407300000002 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.377851+0000", 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002167141 2024-04-26T04:33:25.580 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.762292+0000", 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.38444051499999998 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.762397+0000", 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000105068 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.581 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:258 3.1s0 3:8c067cfd:::smithi17829308-21:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:29.892806+0000", 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 55.668871613999997, 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0716308809999999, 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 258 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.582 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.892806+0000", 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.892806+0000", 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.892807+0000", 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1999999999999999e-07 2024-04-26T04:33:25.583 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.892810+0000", 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0939999999999999e-06 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.892820+0000", 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0236e-05 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.892828+0000", 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.7670000000000003e-06 2024-04-26T04:33:25.584 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.057801+0000", 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.16497324199999999 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.058058+0000", 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000256879 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.560399+0000", 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50234152799999998 2024-04-26T04:33:25.585 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.560550+0000", 2024-04-26T04:33:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015108100000000001 2024-04-26T04:33:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.964437+0000", 2024-04-26T04:33:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.40388633400000001 2024-04-26T04:33:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.586 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:260 3.1s0 3:8bd869b7:::smithi17829308-2:head [append 0~376832] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:29.993865+0000", 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 55.567812826999997, 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.76692751000000003, 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 260 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.587 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.993865+0000", 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.993865+0000", 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.993866+0000", 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6330000000000001e-06 2024-04-26T04:33:25.588 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994355+0000", 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00048898300000000002 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994380+0000", 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4507000000000001e-05 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994392+0000", 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2496e-05 2024-04-26T04:33:25.589 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.161949+0000", 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.16755651699999999 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.162219+0000", 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00026998000000000001 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.164138+0000", 2024-04-26T04:33:25.590 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001919125 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.266155+0000", 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.102017224 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.760676+0000", 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49452143999999998 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.760792+0000", 2024-04-26T04:33:25.591 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000115605 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:261 3.1s0 3:8bd869b7:::smithi17829308-2:head [setxattr _header (55) in=7b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:29.994409+0000", 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 55.567268603999999, 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.76693917899999997, 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.592 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 261 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994409+0000", 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994409+0000", 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.593 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994410+0000", 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.8599999999999996e-07 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994419+0000", 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.037e-06 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994430+0000", 2024-04-26T04:33:25.594 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0794e-05 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:29.994438+0000", 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9759999999999995e-06 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.370941+0000", 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.376503531 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.595 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.371218+0000", 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027671299999999999 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.372480+0000", 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.001261664 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.377847+0000", 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0053671650000000001 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.596 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.761216+0000", 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.38336967199999999 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:30.761348+0000", 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000131641 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:338 3.7s0 3:e1f05b66:::smithi17829308-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.147909+0000", 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.413768701999999, 2024-04-26T04:33:25.597 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.73236990000000002, 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 338 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.147909+0000", 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.598 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.147909+0000", 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.147910+0000", 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5239999999999999e-06 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.147918+0000", 2024-04-26T04:33:25.599 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.3599999999999998e-06 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.147940+0000", 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.247e-05 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.147957+0000", 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7374e-05 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.436335+0000", 2024-04-26T04:33:25.600 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.28837752999999999 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.436482+0000", 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014667 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.644516+0000", 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20803428900000001 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.644756+0000", 2024-04-26T04:33:25.601 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00023993199999999999 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.880279+0000", 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.235522751 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:344 3.bs0 3:df125fb3:::smithi17829308-41:head [copy-from ver 6] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.180126+0000", 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.381551723999998, 2024-04-26T04:33:25.602 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.76992672399999995, 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 344 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180126+0000", 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.603 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180126+0000", 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180130+0000", 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0969999999999999e-06 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180148+0000", 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7880000000000002e-05 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.604 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180183+0000", 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.5612999999999999e-05 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180200+0000", 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6529999999999999e-05 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180372+0000", 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017194500000000001 2024-04-26T04:33:25.605 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.180532+0000", 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016060400000000001 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.647255+0000", 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.46672253299999999 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.649209+0000", 2024-04-26T04:33:25.606 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0019545180000000001 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.949989+0000", 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.300779095 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.950052+0000", 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3909000000000001e-05 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.607 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:358 3.1s0 3:8f617591:::smithi17829308-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:35.339814+0000", 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 50.221863517999999, 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.112259364, 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 358 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.608 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339814+0000", 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339814+0000", 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339815+0000", 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.2300000000000001e-07 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.609 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339817+0000", 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8729999999999999e-06 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339827+0000", 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2669999999999995e-06 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339834+0000", 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0539999999999997e-06 2024-04-26T04:33:25.610 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339863+0000", 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.904e-05 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:35.339867+0000", 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9779999999999999e-06 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053012+0000", 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.71314480999999996 2024-04-26T04:33:25.611 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.053102+0000", 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.0791999999999994e-05 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:36.452073+0000", 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39897092699999998 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.612 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:373 3.7s0 3:e1f05b66:::smithi17829308-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.411932+0000", 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.149744976999997, 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.80901803500000002, 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 373 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.613 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.411932+0000", 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.411932+0000", 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.411933+0000", 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9700000000000003e-07 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.614 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.411934+0000", 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6169999999999999e-06 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.411940+0000", 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7239999999999997e-06 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.512013+0000", 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10007334900000001 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.615 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.515655+0000", 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0036413909999999999 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.515734+0000", 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.9039e-05 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.919808+0000", 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.40407443100000001 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.616 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.919954+0000", 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000146081 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.220950+0000", 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.30099610599999999 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.617 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:448 3.1s0 3:8bd869b7:::smithi17829308-2:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:46.262520+0000", 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 39.299156986, 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.716085101, 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 448 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.618 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.262520+0000", 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.262520+0000", 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.262521+0000", 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.4499999999999997e-07 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.619 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.262523+0000", 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.407e-06 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.262531+0000", 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5100000000000001e-06 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.262535+0000", 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6380000000000003e-06 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.620 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.572760+0000", 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.31022430499999998 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.572940+0000", 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000180667 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.777913+0000", 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20497271 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.621 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.778070+0000", 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015688 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.978606+0000", 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20053553900000001 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.622 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:600 3.7s0 3:ee767cea:::smithi17829308-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:55.877365+0000", 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 29.684312522999999, 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1179588549999999, 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 600 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.623 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877365+0000", 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877365+0000", 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877367+0000", 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9599999999999999e-06 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.624 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877378+0000", 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1043000000000001e-05 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877401+0000", 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2974000000000001e-05 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877414+0000", 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.279e-05 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.625 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.085975+0000", 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.208561152 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.086223+0000", 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024830400000000003 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.589716+0000", 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50349285499999996 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.626 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.589840+0000", 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012359 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.995324+0000", 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.405484187 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:601 3.7s0 3:e1054bf1:::smithi17829308-37:head [read 0~606208,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:25.627 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:55.877432+0000", 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 29.684245699999998, 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.81405845700000001, 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 601 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.628 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877432+0000", 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877432+0000", 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877433+0000", 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.6000000000000002e-07 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.629 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877438+0000", 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.784e-06 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877452+0000", 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4056000000000001e-05 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877461+0000", 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.5739999999999996e-06 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.630 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.086318+0000", 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20885710599999999 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.086752+0000", 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00043374 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.691490+0000", 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.60473833700000001 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.631 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:602 3.7s0 3:e1054bf1:::smithi17829308-37:head [read 0~606208,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:55.877473+0000", 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 29.684204875999999, 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.81550703000000002, 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 602 2024-04-26T04:33:25.632 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877473+0000", 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877473+0000", 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.633 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877473+0000", 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6599999999999995e-07 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877479+0000", 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.9939999999999997e-06 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877492+0000", 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2353e-05 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.634 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877499+0000", 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.4399999999999999e-06 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.087707+0000", 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.21020797299999999 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.087955+0000", 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00024785699999999998 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.635 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.692980+0000", 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.60502464700000003 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:603 3.7s0 3:e1054bf1:::smithi17829308-37:head [read 0~606208,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:55.877509+0000", 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 29.684168892999999, 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0172811070000001, 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.636 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 603 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877509+0000", 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.637 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877509+0000", 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877509+0000", 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.6700000000000003e-07 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877513+0000", 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7859999999999998e-06 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.638 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.877525+0000", 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.205e-05 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:55.977632+0000", 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100106602 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.089944+0000", 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.112312382 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.639 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.090218+0000", 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027392999999999997 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:56.894790+0000", 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.80457159 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:720 3.bs0 3:d909d6e5:::smithi17829308-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e66)", 2024-04-26T04:33:25.640 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:01.682079+0000", 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 23.879598408, 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.92979241599999995, 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 720 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.682079+0000", 2024-04-26T04:33:25.641 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.682079+0000", 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.682079+0000", 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.1199999999999999e-07 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.642 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.682081+0000", 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3740000000000001e-06 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.682085+0000", 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.0609999999999997e-06 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:01.782170+0000", 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100084801 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.643 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.396384+0000", 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.61421400199999998 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.396662+0000", 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027891 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.611871+0000", 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.21520895600000001 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.644 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:814 3.bs0 3:d909d6e5:::smithi17829308-49:head [append 0~0] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:07.275388+0000", 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 18.286289318000001, 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.75181568200000004, 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 814 2024-04-26T04:33:25.645 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275388+0000", 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275388+0000", 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.646 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275390+0000", 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2180000000000002e-06 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275399+0000", 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.7409999999999998e-06 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275421+0000", 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1926999999999999e-05 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.647 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275434+0000", 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3355e-05 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.415905+0000", 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.14047053800000001 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "op must wait for map", 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.415910+0000", 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.395e-06 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.648 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.516958+0000", 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10104777299999999 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.517130+0000", 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017210000000000001 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.518464+0000", 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0013335510000000001 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.649 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.521142+0000", 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0026784700000000001 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.027069+0000", 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.50592662799999999 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.027204+0000", 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134986 2024-04-26T04:33:25.650 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:815 3.bs0 3:d909d6e5:::smithi17829308-49:head [rollback 14] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:07.275463+0000", 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 18.286214235999999, 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.85294690699999998, 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.651 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 815 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275463+0000", 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275463+0000", 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.652 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275464+0000", 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0900000000000001e-07 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275467+0000", 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0819999999999999e-06 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275474+0000", 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6619999999999999e-06 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.653 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275480+0000", 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3820000000000001e-06 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.724313+0000", 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.44883297900000002 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.724463+0000", 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014975000000000001 2024-04-26T04:33:25.654 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.725424+0000", 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00096111600000000003 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.127662+0000", 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.40223767500000002 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.128209+0000", 2024-04-26T04:33:25.655 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00054786899999999998 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.128410+0000", 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000200683 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:816 3.bs0 3:d909d6e5:::smithi17829308-49:head [append 0~0] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:07.275523+0000", 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 18.286154324999998, 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0583476979999999, 2024-04-26T04:33:25.656 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 816 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275523+0000", 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.657 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275523+0000", 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275524+0000", 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.8500000000000002e-07 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275526+0000", 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4600000000000002e-06 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.658 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275532+0000", 2024-04-26T04:33:25.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.1260000000000003e-06 2024-04-26T04:33:25.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:07.275538+0000", 2024-04-26T04:33:25.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.0260000000000004e-06 2024-04-26T04:33:25.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.659 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.659 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.659 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.230416+0000", 2024-04-26T04:33:25.659 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.954877592 2024-04-26T04:33:25.659 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.661 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.661 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.661 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.230595+0000", 2024-04-26T04:33:25.661 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00017954700000000001 2024-04-26T04:33:25.661 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.231448+0000", 2024-04-26T04:33:25.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00085291700000000002 2024-04-26T04:33:25.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.232431+0000", 2024-04-26T04:33:25.662 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00098273900000000005 2024-04-26T04:33:25.662 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.662 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.662 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.662 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.333824+0000", 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101392836 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:08.333871+0000", 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.6969999999999999e-05 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:988 3.1s0 3:8bd869b7:::smithi17829308-2:head [create,setxattr `\\SSSfaPsYfUrUsdSgQgPSapRRms^pqX[osPqdeo (1122) in=40b,setxattr [ (14664) in=1b,setxattr ]_ZacOjR\\V[PolVkSemZdQgglWneOb`chhho^qsi (1365) in=40b,setxattr dlc^[PchQtttbUqnWU```dtpV[iqhiYUtii\\YjTS (1049) in=40b,setxattr pU[nrRrZQQalsblk`ltfaZqrOram[s]cbdhqjSOr (14664) in=40b,setxattr NsheYe]]`bSVTqp\\eWWl (14664) in=20b,setxattr gscckRPkoUNsYbQjkjm_XOsRZllQlNYf^Q`altka (1894) in=40b,setxattr WsNgSgepYqnlrSUek[c\\mS`YlVcq]aXNdhNZqOU_ (803) in=40b,setxattr r\\eN\\d_t[n`tS (14664) in=13b,setxattr g[QgabsXePRNsPqnNqdiZoTaT_]derTZaajrtZPU (1086) in=40b,setxattr SXZemNXWtQUden_WetNWPtrXbjPZYcPXbbU`YSQb (138) in=40b,setxattr OoOSrPaT_VZXQdOh]`[is_WOWeqkpXa]bi\\epXSP (1226) in=40b,setxattr SiXkffqeiTaTiRYVhZeeSeb\\Qkmm^]UseelOaZ]q (1715) in=40b,setxattr nbg_sNYksteVOZjdrW\\P\\ebkpgXci`^pq_nfcXjr (243) in=40b,setxattr ^WgZ\\tgTZprq`o\\YXUtkYak[tfSW[aotVZesNaNV (735) in=40b,setxattr Zr]jhehfqZbYohtW_kqUhRe[NotmVdcfT[atapt] (1683) in=40b,setxattr ^rYSPceWRnj_nVWePZgm_k]WXkOPTd]Utragh]on (630) in=40b,setxattr _[UVkTkSmaX`mfTX_Qp^k^a\\clWeYeWhNkY_OWn\\ (6736) in=40b,setxattr ]p (14664) in=2b,setxattr ^Rtio`SsUNTPt_daN^XiUZbhTVgfa`_SV`famqjr (1927) in=40b] snapc 1a=[1a,19,16] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:23.998881+0000", 2024-04-26T04:33:25.663 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.562796394, 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.113800803, 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 988 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:23.998881+0000", 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.664 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:23.998881+0000", 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:23.998884+0000", 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.106e-06 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:23.999118+0000", 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000234083 2024-04-26T04:33:25.665 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:25.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:23.999152+0000", 2024-04-26T04:33:25.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3939999999999997e-05 2024-04-26T04:33:25.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:25.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:23.999168+0000", 2024-04-26T04:33:25.666 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6180000000000001e-05 2024-04-26T04:33:25.666 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.666 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.666 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.666 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.097247+0000", 2024-04-26T04:33:25.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.098078735 2024-04-26T04:33:25.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:25.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.097610+0000", 2024-04-26T04:33:25.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00036330999999999999 2024-04-26T04:33:25.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.667 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.667 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:25.667 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.805603+0000", 2024-04-26T04:33:25.667 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.70799267499999996 2024-04-26T04:33:25.667 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.805839+0000", 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000235469 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.807160+0000", 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0013211950000000001 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.957594+0000", 2024-04-26T04:33:25.668 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.15043453600000001 2024-04-26T04:33:25.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:25.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.112099+0000", 2024-04-26T04:33:25.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.154504487 2024-04-26T04:33:25.669 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:25.669 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:25.669 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:25.669 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.112682+0000", 2024-04-26T04:33:25.669 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00058308700000000004 2024-04-26T04:33:25.669 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.670 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.670 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.670 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:25.670 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:25.670 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:25.671 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-26T04:33:25.672 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:25.671+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:25.772 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:25.765+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:25.793 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:25.793 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:33:25.793 INFO:teuthology.orchestra.run.smithi012.stdout: "num_ops": 0 2024-04-26T04:33:25.793 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:25.808 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-26T04:33:25.873 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:25.870+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:25.902 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-26T04:33:25.974 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:25.974 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [], 2024-04-26T04:33:25.974 INFO:teuthology.orchestra.run.smithi012.stdout: "complaint_time": 30, 2024-04-26T04:33:25.974 INFO:teuthology.orchestra.run.smithi012.stdout: "num_blocked_ops": 0 2024-04-26T04:33:25.974 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:25.975 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:25.969+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:25.990 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-26T04:33:26.044 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:33:26.045 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:33:26.066 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:33:26.066 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:33:26.076 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:26.069+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:26.087 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:33:26.087 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:33:26.108 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:33:26.109 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:33:26.125 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:33:26.126 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:33:26.146 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:33:26.146 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:33:26.158 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:33:26.158 INFO:teuthology.orchestra.run.smithi012.stdout: "size": 20, 2024-04-26T04:33:26.158 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 600, 2024-04-26T04:33:26.158 INFO:teuthology.orchestra.run.smithi012.stdout: "ops": [ 2024-04-26T04:33:26.159 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.159 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:375 3.4s0 3:26eb43ca:::smithi17829308-16:head [append 0~507904] snapc 8=[8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:33:26.159 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.427459+0000", 2024-04-26T04:33:26.159 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.726367717000002, 2024-04-26T04:33:26.159 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.60387316999999996, 2024-04-26T04:33:26.160 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.160 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.160 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:26.160 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.160 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.160 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.161 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 375 2024-04-26T04:33:26.161 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.161 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.161 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.161 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.161 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.427459+0000", 2024-04-26T04:33:26.162 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.163 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.163 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.163 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.163 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.427459+0000", 2024-04-26T04:33:26.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.164 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.164 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.164 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.164 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.427460+0000", 2024-04-26T04:33:26.164 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0499999999999999e-06 2024-04-26T04:33:26.165 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.165 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.165 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.165 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.428039+0000", 2024-04-26T04:33:26.165 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00057915600000000005 2024-04-26T04:33:26.166 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.166 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.166 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.166 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.428054+0000", 2024-04-26T04:33:26.166 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4941e-05 2024-04-26T04:33:26.167 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.167 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.167 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.167 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.428061+0000", 2024-04-26T04:33:26.167 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8129999999999998e-06 2024-04-26T04:33:26.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.168 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.168 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.168 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430497+0000", 2024-04-26T04:33:26.168 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002436405 2024-04-26T04:33:26.168 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.169 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:26.169 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430504+0000", 2024-04-26T04:33:26.169 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.0299999999999996e-06 2024-04-26T04:33:26.169 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.169 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.170 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.170 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530949+0000", 2024-04-26T04:33:26.170 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100444931 2024-04-26T04:33:26.170 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.170 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.531063+0000", 2024-04-26T04:33:26.171 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0001138 2024-04-26T04:33:26.171 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.171 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.171 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:26.171 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.531997+0000", 2024-04-26T04:33:26.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00093402000000000003 2024-04-26T04:33:26.172 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.172 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.172 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:26.172 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.533975+0000", 2024-04-26T04:33:26.172 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0019779659999999998 2024-04-26T04:33:26.173 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:26.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.031211+0000", 2024-04-26T04:33:26.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49723574799999998 2024-04-26T04:33:26.174 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.174 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.174 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.174 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.031332+0000", 2024-04-26T04:33:26.174 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012131 2024-04-26T04:33:26.174 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.174 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.174 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:376 3.4s0 3:26eb43ca:::smithi17829308-16:head [append 0~262144] snapc 8=[8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.430172+0000", 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.723655145999999, 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.70162850899999996, 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 376 2024-04-26T04:33:26.175 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.176 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430172+0000", 2024-04-26T04:33:26.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.176 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430172+0000", 2024-04-26T04:33:26.176 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.176 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.176 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.176 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430173+0000", 2024-04-26T04:33:26.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.1200000000000001e-07 2024-04-26T04:33:26.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430402+0000", 2024-04-26T04:33:26.177 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00022931899999999999 2024-04-26T04:33:26.177 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.177 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.177 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.177 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430418+0000", 2024-04-26T04:33:26.178 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6401000000000001e-05 2024-04-26T04:33:26.179 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.179 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.179 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.179 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430431+0000", 2024-04-26T04:33:26.179 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2278e-05 2024-04-26T04:33:26.180 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.180 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.180 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.180 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430523+0000", 2024-04-26T04:33:26.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.2398999999999994e-05 2024-04-26T04:33:26.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:26.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430526+0000", 2024-04-26T04:33:26.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.9809999999999999e-06 2024-04-26T04:33:26.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.181 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.181 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.181 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.532597+0000", 2024-04-26T04:33:26.181 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.102070991 2024-04-26T04:33:26.181 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.532697+0000", 2024-04-26T04:33:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000100453 2024-04-26T04:33:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.533465+0000", 2024-04-26T04:33:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00076714800000000005 2024-04-26T04:33:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.182 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.534877+0000", 2024-04-26T04:33:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00141208 2024-04-26T04:33:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.131673+0000", 2024-04-26T04:33:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.59679601000000004 2024-04-26T04:33:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.131800+0000", 2024-04-26T04:33:26.183 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000127537 2024-04-26T04:33:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:377 3.4s0 3:26eb43ca:::smithi17829308-16:head [setxattr _header (56) in=7b] snapc 8=[8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:33:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.430444+0000", 2024-04-26T04:33:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.723382356000002, 2024-04-26T04:33:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.70171558099999998, 2024-04-26T04:33:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.184 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 377 2024-04-26T04:33:26.185 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.186 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.186 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.186 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430444+0000", 2024-04-26T04:33:26.186 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.186 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.186 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.187 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.187 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430444+0000", 2024-04-26T04:33:26.187 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.187 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.187 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.188 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.188 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430445+0000", 2024-04-26T04:33:26.188 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.68e-07 2024-04-26T04:33:26.188 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.188 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430450+0000", 2024-04-26T04:33:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.493e-06 2024-04-26T04:33:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.189 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430463+0000", 2024-04-26T04:33:26.190 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.3378e-05 2024-04-26T04:33:26.190 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.190 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.190 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430470+0000", 2024-04-26T04:33:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1740000000000003e-06 2024-04-26T04:33:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.191 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.504874+0000", 2024-04-26T04:33:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.074404153000000001 2024-04-26T04:33:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.504878+0000", 2024-04-26T04:33:26.192 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.9759999999999997e-06 2024-04-26T04:33:26.193 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.193 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.193 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.193 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.534051+0000", 2024-04-26T04:33:26.193 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.029172849000000001 2024-04-26T04:33:26.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.194 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.194 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.534150+0000", 2024-04-26T04:33:26.194 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.857e-05 2024-04-26T04:33:26.194 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.194 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:26.195 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.534617+0000", 2024-04-26T04:33:26.195 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00046738200000000002 2024-04-26T04:33:26.195 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.195 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.195 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.535807+0000", 2024-04-26T04:33:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011894379999999999 2024-04-26T04:33:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.132031+0000", 2024-04-26T04:33:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.59622439299999996 2024-04-26T04:33:26.196 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.197 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.197 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.197 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.132160+0000", 2024-04-26T04:33:26.197 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000129107 2024-04-26T04:33:26.197 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.197 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.197 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.198 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.198 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.198 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:378 3.4s0 3:26eb43ca:::smithi17829308-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:33:26.198 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.430481+0000", 2024-04-26T04:33:26.198 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.723345442999999, 2024-04-26T04:33:26.198 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1057942590000001, 2024-04-26T04:33:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.199 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 378 2024-04-26T04:33:26.200 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.200 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.200 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.200 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.200 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430481+0000", 2024-04-26T04:33:26.200 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430481+0000", 2024-04-26T04:33:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.201 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430482+0000", 2024-04-26T04:33:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.4000000000000002e-07 2024-04-26T04:33:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430484+0000", 2024-04-26T04:33:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.52e-06 2024-04-26T04:33:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.430493+0000", 2024-04-26T04:33:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.9139999999999997e-06 2024-04-26T04:33:26.202 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530567+0000", 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10007361300000001 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.535742+0000", 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0051753149999999998 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.535800+0000", 2024-04-26T04:33:26.203 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.7997000000000001e-05 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.138278+0000", 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.60247765099999995 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.138452+0000", 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000173618 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.536276+0000", 2024-04-26T04:33:26.204 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.39782409099999999 2024-04-26T04:33:26.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.205 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.205 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.205 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.205 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.205 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:379 3.4s0 3:25070d22:::smithi17829308-10:head [copy-from ver 2] snapc 8=[8,7,6,5] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:33:26.205 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:39.530582+0000", 2024-04-26T04:33:26.205 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 46.623244434, 2024-04-26T04:33:26.205 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1058113540000001, 2024-04-26T04:33:26.205 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.205 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.205 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:26.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.206 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.206 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 379 2024-04-26T04:33:26.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.206 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.206 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530582+0000", 2024-04-26T04:33:26.206 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.206 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.206 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.206 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530582+0000", 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530584+0000", 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.314e-06 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530589+0000", 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.6189999999999999e-06 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.207 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.530598+0000", 2024-04-26T04:33:26.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.2979999999999992e-06 2024-04-26T04:33:26.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.630723+0000", 2024-04-26T04:33:26.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10012523199999999 2024-04-26T04:33:26.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.208 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.208 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.208 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.930085+0000", 2024-04-26T04:33:26.208 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.29936250800000003 2024-04-26T04:33:26.208 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:39.930252+0000", 2024-04-26T04:33:26.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000166665 2024-04-26T04:33:26.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:26.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.136077+0000", 2024-04-26T04:33:26.209 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.20582483400000001 2024-04-26T04:33:26.209 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.209 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.209 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:26.209 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.139888+0000", 2024-04-26T04:33:26.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00381131 2024-04-26T04:33:26.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:26.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.636327+0000", 2024-04-26T04:33:26.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.49643924299999997 2024-04-26T04:33:26.210 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.210 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.210 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.210 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:40.636394+0000", 2024-04-26T04:33:26.210 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.6330999999999999e-05 2024-04-26T04:33:26.211 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.211 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.211 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.211 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.211 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.211 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:435 3.4s0 3:25070d22:::smithi17829308-10:head [append 0~638976] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:26.211 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:46.214523+0000", 2024-04-26T04:33:26.211 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 39.939303901000002, 2024-04-26T04:33:26.211 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.52591575599999996, 2024-04-26T04:33:26.212 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.212 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.212 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:26.212 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.212 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.212 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.213 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 435 2024-04-26T04:33:26.213 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.213 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.213 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.213 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.213 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.214523+0000", 2024-04-26T04:33:26.213 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.214 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.214523+0000", 2024-04-26T04:33:26.214 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.214 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.214 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.214 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.214526+0000", 2024-04-26T04:33:26.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.2559999999999998e-06 2024-04-26T04:33:26.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.215982+0000", 2024-04-26T04:33:26.215 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0014555219999999999 2024-04-26T04:33:26.215 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.215 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.215 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.215 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.216021+0000", 2024-04-26T04:33:26.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8917e-05 2024-04-26T04:33:26.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.216047+0000", 2024-04-26T04:33:26.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6503e-05 2024-04-26T04:33:26.216 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.216 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.216 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.216 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.224553+0000", 2024-04-26T04:33:26.216 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0085058909999999998 2024-04-26T04:33:26.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.224734+0000", 2024-04-26T04:33:26.217 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018098100000000001 2024-04-26T04:33:26.217 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.217 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.217 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:26.217 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.226765+0000", 2024-04-26T04:33:26.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002030727 2024-04-26T04:33:26.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.218 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.218 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:26.218 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.318035+0000", 2024-04-26T04:33:26.218 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.091269829999999996 2024-04-26T04:33:26.218 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.219 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:26.219 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.740201+0000", 2024-04-26T04:33:26.219 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.42216641199999999 2024-04-26T04:33:26.219 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.219 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.220 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.220 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.740439+0000", 2024-04-26T04:33:26.220 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000237717 2024-04-26T04:33:26.220 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.220 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.220 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.221 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.221 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.221 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:436 3.4s0 3:25070d22:::smithi17829308-10:head [append 0~622592] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:26.221 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:46.223140+0000", 2024-04-26T04:33:26.221 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 39.930686586, 2024-04-26T04:33:26.221 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.51794441999999996, 2024-04-26T04:33:26.221 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.221 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.221 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:26.221 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.222 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.222 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.222 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 436 2024-04-26T04:33:26.222 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.222 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.222 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.222 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.222 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.223140+0000", 2024-04-26T04:33:26.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.223 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.223 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.223 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.223 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.223140+0000", 2024-04-26T04:33:26.223 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.224 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.224 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.223142+0000", 2024-04-26T04:33:26.224 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.08e-06 2024-04-26T04:33:26.224 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.224 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.224304+0000", 2024-04-26T04:33:26.225 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0011613369999999999 2024-04-26T04:33:26.225 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.225 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.225 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.225 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.224338+0000", 2024-04-26T04:33:26.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3930000000000002e-05 2024-04-26T04:33:26.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.224361+0000", 2024-04-26T04:33:26.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3904e-05 2024-04-26T04:33:26.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.226 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.226 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.228899+0000", 2024-04-26T04:33:26.226 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0045370539999999996 2024-04-26T04:33:26.226 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.226 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.229064+0000", 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016549699999999999 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.230844+0000", 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0017795230000000001 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.318040+0000", 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.08719644 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.227 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.740869+0000", 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.422828914 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.741085+0000", 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00021574100000000001 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.228 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:439 3.4s0 3:25070d22:::smithi17829308-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:32:46.225576+0000", 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 39.928250374999998, 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.62140023300000002, 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 439 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.229 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.225576+0000", 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.225576+0000", 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.225578+0000", 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2500000000000001e-06 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.230 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.225585+0000", 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.1040000000000001e-06 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.225611+0000", 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.582e-05 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.225627+0000", 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.6826000000000001e-05 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.231 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.317734+0000", 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.092106697000000001 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.318009+0000", 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027465699999999998 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.744281+0000", 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.42627229599999999 2024-04-26T04:33:26.232 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.744529+0000", 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000248354 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:32:46.846977+0000", 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.102447229 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.233 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.234 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:723 3.4s0 3:2d4f4f0a:::smithi17829308-44:head [copy-from ver 6] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-26T04:33:26.234 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.299030+0000", 2024-04-26T04:33:26.234 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 23.854796522000001, 2024-04-26T04:33:26.234 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.82259208399999995, 2024-04-26T04:33:26.234 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.234 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.234 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:26.234 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.234 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.234 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.234 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 723 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.299030+0000", 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.299030+0000", 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.235 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.299033+0000", 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.357e-06 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.299043+0000", 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.0443e-05 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.299073+0000", 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.0219999999999999e-05 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.236 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.299095+0000", 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1863000000000001e-05 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.299240+0000", 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014516999999999999 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.299391+0000", 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015024400000000001 2024-04-26T04:33:26.237 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:26.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.919749+0000", 2024-04-26T04:33:26.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.62035839500000001 2024-04-26T04:33:26.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:26.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.927711+0000", 2024-04-26T04:33:26.238 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0079622070000000007 2024-04-26T04:33:26.238 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.238 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.238 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:26.238 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.121511+0000", 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.19380016799999999 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.121622+0000", 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000111017 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:724 3.4s0 3:2d4f4f0a:::smithi17829308-44:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e66)", 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.299169+0000", 2024-04-26T04:33:26.239 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 23.854657638999999, 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.62804005699999998, 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 724 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.299169+0000", 2024-04-26T04:33:26.240 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.299169+0000", 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.299170+0000", 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.004e-06 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.241 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.299175+0000", 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 5.0560000000000004e-06 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.299197+0000", 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1367000000000001e-05 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.399312+0000", 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10011521800000001 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.242 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.399633+0000", 2024-04-26T04:33:26.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032089299999999999 2024-04-26T04:33:26.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:26.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.399798+0000", 2024-04-26T04:33:26.243 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016579400000000001 2024-04-26T04:33:26.243 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.243 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.243 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.243 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.926923+0000", 2024-04-26T04:33:26.245 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.52712415499999998 2024-04-26T04:33:26.245 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.246 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.246 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.246 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.927052+0000", 2024-04-26T04:33:26.246 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00012927300000000001 2024-04-26T04:33:26.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.247 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.247 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.247 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.927209+0000", 2024-04-26T04:33:26.247 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00015729700000000001 2024-04-26T04:33:26.247 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.247 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.247 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.247 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.248 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.248 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:736 3.4s0 3:2b198661:::smithi17829308-4:head [append 0~720896] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:26.248 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.402059+0000", 2024-04-26T04:33:26.248 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 23.751768215999999, 2024-04-26T04:33:26.248 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.52562701300000003, 2024-04-26T04:33:26.248 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.248 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.248 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:26.248 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.248 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.248 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.248 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 736 2024-04-26T04:33:26.248 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.402059+0000", 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.402059+0000", 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.249 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.402061+0000", 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.3269999999999999e-06 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404746+0000", 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0026849080000000002 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404783+0000", 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.7246999999999998e-05 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.250 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404804+0000", 2024-04-26T04:33:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.1302999999999999e-05 2024-04-26T04:33:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.610399+0000", 2024-04-26T04:33:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.205594993 2024-04-26T04:33:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.610560+0000", 2024-04-26T04:33:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0001606 2024-04-26T04:33:26.251 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.612632+0000", 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002072305 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.615324+0000", 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0026921979999999998 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.927573+0000", 2024-04-26T04:33:26.252 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.31224862599999997 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.927686+0000", 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000112506 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:737 3.4s0 3:2b198661:::smithi17829308-4:head [append 0~32768] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.404844+0000", 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 23.748982771000001, 2024-04-26T04:33:26.253 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.52335029499999997, 2024-04-26T04:33:26.254 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.254 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.254 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:26.254 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.254 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.254 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.254 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 737 2024-04-26T04:33:26.254 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.254 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.254 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.254 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.254 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404844+0000", 2024-04-26T04:33:26.254 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404844+0000", 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404846+0000", 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.0229999999999999e-06 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.255 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.404942+0000", 2024-04-26T04:33:26.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.6163999999999996e-05 2024-04-26T04:33:26.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405127+0000", 2024-04-26T04:33:26.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00018465300000000001 2024-04-26T04:33:26.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.256 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.256 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405165+0000", 2024-04-26T04:33:26.256 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.8550000000000002e-05 2024-04-26T04:33:26.256 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.256 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.257 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.257 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.615507+0000", 2024-04-26T04:33:26.257 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.210341792 2024-04-26T04:33:26.257 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.257 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.615668+0000", 2024-04-26T04:33:26.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00016094699999999999 2024-04-26T04:33:26.258 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.258 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.258 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:26.258 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.616508+0000", 2024-04-26T04:33:26.258 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00083959599999999996 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.620598+0000", 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.004090214 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.928093+0000", 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.30749454599999998 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.259 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.928194+0000", 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00010181 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:738 3.4s0 3:2b198661:::smithi17829308-4:head [setxattr _header (56) in=7b] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.405201+0000", 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 23.748626238, 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.52347687200000004, 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.260 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 738 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405201+0000", 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405201+0000", 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.261 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.262 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.262 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.262 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405202+0000", 2024-04-26T04:33:26.262 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.683e-06 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405212+0000", 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.012e-05 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405238+0000", 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.6047e-05 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.263 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405256+0000", 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7427000000000001e-05 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.617442+0000", 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.212186344 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.617738+0000", 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000295474 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.264 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.618737+0000", 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00099954199999999997 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.621173+0000", 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.002435619 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.928543+0000", 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.30737000199999998 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.265 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.928677+0000", 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000134614 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:739 3.4s0 3:2b198661:::smithi17829308-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.405283+0000", 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 23.748544058, 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.71600204000000001, 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.266 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 739 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405283+0000", 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.267 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405283+0000", 2024-04-26T04:33:26.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.268 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.268 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405284+0000", 2024-04-26T04:33:26.268 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.353e-06 2024-04-26T04:33:26.268 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.268 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405291+0000", 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.8600000000000004e-06 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405310+0000", 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.946e-05 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405328+0000", 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.7093999999999999e-05 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.269 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.619429+0000", 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.21410157299999999 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.619701+0000", 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000272 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.028877+0000", 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.40917559199999998 2024-04-26T04:33:26.270 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.029019+0000", 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014231000000000001 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.121285+0000", 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.092265797999999996 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.271 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:747 3.4s0 3:25070d22:::smithi17829308-10:head [copy-from ver 2] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:26.272 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.405354+0000", 2024-04-26T04:33:26.272 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 23.748472905, 2024-04-26T04:33:26.272 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.71655392900000003, 2024-04-26T04:33:26.272 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.272 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.272 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:26.272 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.272 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.272 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.272 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 747 2024-04-26T04:33:26.272 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.272 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.272 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405354+0000", 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405354+0000", 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405355+0000", 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.257e-06 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.273 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405363+0000", 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 7.5730000000000002e-06 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405385+0000", 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2217000000000001e-05 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405400+0000", 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.5136999999999999e-05 2024-04-26T04:33:26.274 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.619948+0000", 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.21454841899999999 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.620219+0000", 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00027095299999999999 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_started", 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.924867+0000", 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.30464720499999998 2024-04-26T04:33:26.275 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "sub_op_committed", 2024-04-26T04:33:26.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.927779+0000", 2024-04-26T04:33:26.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.0029123450000000002 2024-04-26T04:33:26.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "commit_sent", 2024-04-26T04:33:26.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.121812+0000", 2024-04-26T04:33:26.276 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.19403298599999999 2024-04-26T04:33:26.276 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.276 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.276 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.276 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:03.121908+0000", 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 9.5836999999999997e-05 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:748 3.4s0 3:25070d22:::smithi17829308-10:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:02.405421+0000", 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 23.748405570999999, 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.52137614899999996, 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.277 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.278 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.278 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 748 2024-04-26T04:33:26.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.278 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.278 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.278 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405421+0000", 2024-04-26T04:33:26.278 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.278 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.278 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405421+0000", 2024-04-26T04:33:26.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405422+0000", 2024-04-26T04:33:26.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.223e-06 2024-04-26T04:33:26.279 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.279 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.279 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.279 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405429+0000", 2024-04-26T04:33:26.279 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.3470000000000004e-06 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405452+0000", 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.2949999999999999e-05 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.405467+0000", 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.4922999999999999e-05 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.280 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.620704+0000", 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.215237506 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.620851+0000", 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00014702000000000001 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.926196+0000", 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.30534483000000001 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.281 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.926473+0000", 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.000277137 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:02.926797+0000", 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00032421300000000001 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "osd_op(client.4670.0:993 3.4s0 3:2b198661:::smithi17829308-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e88)", 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:24.339667+0000", 2024-04-26T04:33:26.282 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.814159334, 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.117793187, 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "started", 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: "client_info": { 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: "client": "client.4670", 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: "tid": 993 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.339667+0000", 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.283 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.339667+0000", 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.339669+0000", 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.9599999999999999e-06 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.339681+0000", 2024-04-26T04:33:26.284 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1613999999999999e-05 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.339709+0000", 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.8334e-05 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.339735+0000", 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5743000000000001e-05 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.415969+0000", 2024-04-26T04:33:26.285 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.076233565000000003 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.416413+0000", 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00044424999999999998 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.518293+0000", 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.101879924 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "started", 2024-04-26T04:33:26.286 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.518604+0000", 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.00031108500000000001 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.457461+0000", 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.93885671199999998 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpRead(3.4s0 89/30 ECSubRead(tid=135, to_read={3:2b198661:::smithi17829308-4:head=0,4096,0}, subchunks={3:2b198661:::smithi17829308-4:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:24.519181+0000", 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.63464572, 2024-04-26T04:33:26.287 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.63709345900000003, 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.519181+0000", 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.519181+0000", 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.288 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.519181+0000", 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.519181+0000", 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.519206+0000", 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.4629000000000001e-05 2024-04-26T04:33:26.289 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.619299+0000", 2024-04-26T04:33:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10009301 2024-04-26T04:33:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.156186+0000", 2024-04-26T04:33:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.53688715799999998 2024-04-26T04:33:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.290 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.156275+0000", 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 8.8661999999999999e-05 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpReadReply(3.4s0 89/30 ECSubReadReply(tid=135, attrs_read=0))", 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:24.519659+0000", 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.6341682230000001, 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.73686818399999998, 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.291 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.519659+0000", 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.519659+0000", 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.519660+0000", 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.677e-06 2024-04-26T04:33:26.292 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.519685+0000", 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.497e-05 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.519711+0000", 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 2.5695999999999999e-05 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.619804+0000", 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.100092747 2024-04-26T04:33:26.293 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.256494+0000", 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.63668996499999997 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.256527+0000", 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 3.3129000000000001e-05 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.294 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: "description": "MOSDECSubOpReadReply(3.4s0 89/30 ECSubReadReply(tid=135, attrs_read=0))", 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: "initiated_at": "2024-04-26T04:33:24.681190+0000", 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: "age": 1.4726369130000001, 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.57527657799999998, 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: "continuous": false, 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: "type_data": { 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: "flag_point": "reached pg", 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: "events": [ 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "initiated", 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.681190+0000", 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.295 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "header_read", 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.681190+0000", 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "throttled", 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.681191+0000", 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 6.5700000000000002e-07 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "all_read", 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.681203+0000", 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.2379e-05 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.296 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "dispatched", 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.681214+0000", 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 1.1432e-05 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "queued_for_pg", 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:24.781313+0000", 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.10009826600000001 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "reached_pg", 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.256417+0000", 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 0.475104209 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: }, 2024-04-26T04:33:26.297 INFO:teuthology.orchestra.run.smithi012.stdout: { 2024-04-26T04:33:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: "event": "done", 2024-04-26T04:33:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: "time": "2024-04-26T04:33:25.256466+0000", 2024-04-26T04:33:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: "duration": 4.9635000000000002e-05 2024-04-26T04:33:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: } 2024-04-26T04:33:26.298 INFO:teuthology.orchestra.run.smithi012.stdout: ] 2024-04-26T04:33:26.298 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:33:26.299 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:33:26.299 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:33:26.300 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:33:26.303 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:26.297+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:26.334 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:33:26.334 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:33:26.361 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:33:26.361 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:33:26.383 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:33:26.383 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:33:26.401 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:26.397+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:26.402 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:33:26.402 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:33:26.424 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:33:26.425 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:33:26.448 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:33:26.449 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:33:26.459 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:26.460 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:26.472 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:33:26.472 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:33:26.474 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-26T04:33:26.502 INFO:tasks.ceph.osd.13.smithi113.stderr:2024-04-26T04:33:26.500+0000 7ffac0c19640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25076) UID: 0 2024-04-26T04:33:26.603 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:26.600+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:26.665 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:26.665 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:26.680 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-26T04:33:26.704 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:26.703+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:26.804 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:33:26.801+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:33:26.805 INFO:teuthology.orchestra.run.smithi012.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:26.805 INFO:teuthology.orchestra.run.smithi012.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:26.820 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T04:33:26.906 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:26.901+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:26.952 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:26.952 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:26.966 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-26T04:33:27.007 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:27.007+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:27.084 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:27.084 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:27.098 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-26T04:33:27.108 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:27.101+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:27.209 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:27.205+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:27.217 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:27.217 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:27.231 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-26T04:33:27.311 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:27.305+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:27.351 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:27.351 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:27.365 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-26T04:33:27.411 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-04-26T04:33:27.405+0000 7f2076ab6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25078) UID: 0 2024-04-26T04:33:27.486 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:27.486 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:27.500 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-26T04:33:27.512 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:27.509+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:27.614 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:27.615+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:27.627 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:27.627 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:27.642 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-26T04:33:27.714 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:27.709+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:27.764 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:27.765 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:27.779 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-26T04:33:27.815 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:27.809+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:27.896 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:27.896 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:27.909 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-26T04:33:27.915 INFO:tasks.ceph.osd.10.smithi137.stderr:2024-04-26T04:33:27.915+0000 7fe45248b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25091) UID: 0 2024-04-26T04:33:28.016 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:28.013+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:28.017 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:28.017 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:28.030 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-26T04:33:28.046 INFO:tasks.rados.rados.0.smithi178.stdout:386: finishing rollback tid 1 to smithi17829308-21 2024-04-26T04:33:28.047 INFO:tasks.rados.rados.0.smithi178.stdout:386: finishing rollback tid 2 to smithi17829308-21 2024-04-26T04:33:28.047 INFO:tasks.rados.rados.0.smithi178.stdout:update_object_version oid 21 v 105 (ObjNum 147 snap 23 seq_num 147) dirty exists 2024-04-26T04:33:28.047 INFO:tasks.rados.rados.0.smithi178.stdout:386: done (0 left) 2024-04-26T04:33:28.057 INFO:tasks.rados.rados.0.smithi178.stderr:0 errors. 2024-04-26T04:33:28.058 INFO:tasks.rados.rados.0.smithi178.stderr: 2024-04-26T04:33:28.069 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T04:33:28.118 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:28.116+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:28.150 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:28.150 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:28.164 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-26T04:33:28.219 INFO:tasks.ceph.osd.3.smithi178.stderr:2024-04-26T04:33:28.217+0000 7f8066800640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25350) UID: 0 2024-04-26T04:33:28.303 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-26T04:33:28.320 INFO:tasks.ceph.osd.15.smithi178.stderr:2024-04-26T04:33:28.317+0000 7fd3465ff640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25356) UID: 0 2024-04-26T04:33:28.322 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:28.322 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:28.335 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-26T04:33:28.386 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:33:28.386 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":92,"fsid":"12c27fbe-0161-48d0-88ec-42a226685437","created":"2024-04-26T04:31:56.451062+0000","modified":"2024-04-26T04:33:27.377151+0000","last_up_change":"2024-04-26T04:32:09.452464+0000","last_in_change":"2024-04-26T04:33:19.176468+0000","flags":"nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5804032,"flags_set":["nodeep-scrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:32:07.426875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":0.96524238586425781,"average_primary_affinity":0.97000002861022949,"average_primary_affinity_weighted":0.97000002861022949}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-26T04:32:11.318957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.96524238586425781,"average_primary_affinity":0.97000002861022949,"average_primary_affinity_weighted":0.97000002861022949}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-04-26T04:32:21.196824+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":7,"min_size":5,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":26,"snap_epoch":89,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"shecprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":16384,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"55a4f254-2c2a-4007-9c52-b89df6fffb69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6817","nonce":1754063127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6819","nonce":1754063127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6823","nonce":1754063127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6821","nonce":1754063127}]},"public_addr":"172.21.15.12:6817/1754063127","cluster_addr":"172.21.15.12:6819/1754063127","heartbeat_back_addr":"172.21.15.12:6823/1754063127","heartbeat_front_addr":"172.21.15.12:6821/1754063127","state":["exists","up"]},{"osd":1,"uuid":"32db4f97-c8ac-481c-9e80-18fc91004298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6809","nonce":3264326676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6811","nonce":3264326676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6817","nonce":3264326676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6813","nonce":3264326676}]},"public_addr":"172.21.15.113:6809/3264326676","cluster_addr":"172.21.15.113:6811/3264326676","heartbeat_back_addr":"172.21.15.113:6817/3264326676","heartbeat_front_addr":"172.21.15.113:6813/3264326676","state":["exists","up"]},{"osd":2,"uuid":"2cd26466-5d59-4599-ba3f-a78b0e3b681f","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6802","nonce":1454475448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6806","nonce":1454475448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6814","nonce":1454475448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6810","nonce":1454475448}]},"public_addr":"172.21.15.137:6802/1454475448","cluster_addr":"172.21.15.137:6806/1454475448","heartbeat_back_addr":"172.21.15.137:6814/1454475448","heartbeat_front_addr":"172.21.15.137:6810/1454475448","state":["exists","up"]},{"osd":3,"uuid":"0945017d-9525-49d6-aa67-6635acd94c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6803","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6805","nonce":2898698985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6810","nonce":2898698985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6817","nonce":2898698985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6815","nonce":2898698985}]},"public_addr":"172.21.15.178:6805/2898698985","cluster_addr":"172.21.15.178:6810/2898698985","heartbeat_back_addr":"172.21.15.178:6817/2898698985","heartbeat_front_addr":"172.21.15.178:6815/2898698985","state":["exists","up"]},{"osd":4,"uuid":"38dc68bf-0a89-4900-9d35-24c70368447a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6809","nonce":3209840823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6811","nonce":3209840823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6815","nonce":3209840823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6813","nonce":3209840823}]},"public_addr":"172.21.15.12:6809/3209840823","cluster_addr":"172.21.15.12:6811/3209840823","heartbeat_back_addr":"172.21.15.12:6815/3209840823","heartbeat_front_addr":"172.21.15.12:6813/3209840823","state":["exists","up"]},{"osd":5,"uuid":"e91cc8a1-57e1-4760-a329-8a60aa752a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6801","nonce":1475637773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6803","nonce":1475637773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6807","nonce":1475637773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6805","nonce":1475637773}]},"public_addr":"172.21.15.113:6801/1475637773","cluster_addr":"172.21.15.113:6803/1475637773","heartbeat_back_addr":"172.21.15.113:6807/1475637773","heartbeat_front_addr":"172.21.15.113:6805/1475637773","state":["exists","up"]},{"osd":6,"uuid":"a1322130-3391-432d-a1ee-f83297885029","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6817","nonce":771338783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6819","nonce":771338783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6823","nonce":771338783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6821","nonce":771338783}]},"public_addr":"172.21.15.137:6817/771338783","cluster_addr":"172.21.15.137:6819/771338783","heartbeat_back_addr":"172.21.15.137:6823/771338783","heartbeat_front_addr":"172.21.15.137:6821/771338783","state":["exists","up"]},{"osd":7,"uuid":"02ee4ca2-9d48-4aff-9641-1823760c7a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6827","nonce":1017152077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6829","nonce":1017152077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6833","nonce":1017152077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6831","nonce":1017152077}]},"public_addr":"172.21.15.178:6827/1017152077","cluster_addr":"172.21.15.178:6829/1017152077","heartbeat_back_addr":"172.21.15.178:6833/1017152077","heartbeat_front_addr":"172.21.15.178:6831/1017152077","state":["exists","up"]},{"osd":8,"uuid":"ea3d5f62-3d68-4824-8556-da8db36553fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6801","nonce":2153129138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6803","nonce":2153129138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6807","nonce":2153129138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6805","nonce":2153129138}]},"public_addr":"172.21.15.12:6801/2153129138","cluster_addr":"172.21.15.12:6803/2153129138","heartbeat_back_addr":"172.21.15.12:6807/2153129138","heartbeat_front_addr":"172.21.15.12:6805/2153129138","state":["exists","up"]},{"osd":9,"uuid":"e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6825","nonce":359931551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6827","nonce":359931551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6831","nonce":359931551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6829","nonce":359931551}]},"public_addr":"172.21.15.113:6825/359931551","cluster_addr":"172.21.15.113:6827/359931551","heartbeat_back_addr":"172.21.15.113:6831/359931551","heartbeat_front_addr":"172.21.15.113:6829/359931551","state":["exists","up"]},{"osd":10,"uuid":"8c4f8df5-465a-486e-9413-1de14e33c119","up":1,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6801","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6803","nonce":1533480861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6807","nonce":1533480861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6815","nonce":1533480861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6811","nonce":1533480861}]},"public_addr":"172.21.15.137:6803/1533480861","cluster_addr":"172.21.15.137:6807/1533480861","heartbeat_back_addr":"172.21.15.137:6815/1533480861","heartbeat_front_addr":"172.21.15.137:6811/1533480861","state":["exists","up"]},{"osd":11,"uuid":"6fe54c2d-0b95-4235-8c22-51230cfaa591","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6819","nonce":1683682957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6821","nonce":1683682957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6825","nonce":1683682957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6823","nonce":1683682957}]},"public_addr":"172.21.15.178:6819/1683682957","cluster_addr":"172.21.15.178:6821/1683682957","heartbeat_back_addr":"172.21.15.178:6825/1683682957","heartbeat_front_addr":"172.21.15.178:6823/1683682957","state":["exists","up"]},{"osd":12,"uuid":"5ddd0f05-fb78-43bb-95ec-3fc7985603b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6825","nonce":2509311840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6827","nonce":2509311840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6831","nonce":2509311840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6829","nonce":2509311840}]},"public_addr":"172.21.15.12:6825/2509311840","cluster_addr":"172.21.15.12:6827/2509311840","heartbeat_back_addr":"172.21.15.12:6831/2509311840","heartbeat_front_addr":"172.21.15.12:6829/2509311840","state":["exists","up"]},{"osd":13,"uuid":"37eb42c1-f29e-49f5-b8f6-035178038e5d","up":1,"in":1,"weight":1,"primary_affinity":0.443878173828125,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6816","nonce":3716239663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6819","nonce":3716239663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6823","nonce":3716239663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6821","nonce":3716239663}]},"public_addr":"172.21.15.113:6816/3716239663","cluster_addr":"172.21.15.113:6819/3716239663","heartbeat_back_addr":"172.21.15.113:6823/3716239663","heartbeat_front_addr":"172.21.15.113:6821/3716239663","state":["exists","up"]},{"osd":14,"uuid":"b2267367-a857-4a5c-a8d1-6a9b23c748f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6825","nonce":1454956132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6827","nonce":1454956132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6831","nonce":1454956132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6829","nonce":1454956132}]},"public_addr":"172.21.15.137:6825/1454956132","cluster_addr":"172.21.15.137:6827/1454956132","heartbeat_back_addr":"172.21.15.137:6831/1454956132","heartbeat_front_addr":"172.21.15.137:6829/1454956132","state":["exists","up"]},{"osd":15,"uuid":"9933b5aa-e091-43b1-b788-5b79695bab11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6804","nonce":314165740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6807","nonce":314165740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6814","nonce":314165740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6809","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6811","nonce":314165740}]},"public_addr":"172.21.15.178:6804/314165740","cluster_addr":"172.21.15.178:6807/314165740","heartbeat_back_addr":"172.21.15.178:6814/314165740","heartbeat_front_addr":"172.21.15.178:6811/314165740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.305255+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:09.218081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":65536,"last_purged_snaps_scrub":"2024-04-26T04:33:08.352200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:13.912651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.276649+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.349822+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.396513+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:25.153786+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:26.506163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:10.178898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":65536,"last_purged_snaps_scrub":"2024-04-26T04:33:20.633419+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.328959+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:19.550387+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:10.407663+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:32:06.290530+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:14.071867+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.a","osds":[3,1,7,13,6,9,8]}],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":15,"to":4},{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"3.1","osds":[4,11,9,5,2147483647,12,13]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":20,"length":1},{"begin":22,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:33:28.409 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:33:28.421 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:28.420+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:28.457 INFO:teuthology.orchestra.run.smithi113.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:28.458 INFO:teuthology.orchestra.run.smithi113.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:28.471 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T04:33:28.493 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-26T04:33:28.521 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:28.520+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:28.603 INFO:teuthology.orchestra.run.smithi137.stdout:ERROR: (22) Invalid argument 2024-04-26T04:33:28.603 INFO:teuthology.orchestra.run.smithi137.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-26T04:33:28.617 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-26T04:33:28.623 INFO:tasks.ceph.osd.7.smithi178.stderr:2024-04-26T04:33:28.617+0000 7fc7f8135640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25352) UID: 0 2024-04-26T04:33:28.642 INFO:teuthology.orchestra.run.smithi012.stdout:osd.0: {} 2024-04-26T04:33:28.642 INFO:teuthology.orchestra.run.smithi012.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-26T04:33:28.678 INFO:teuthology.orchestra.run.smithi012.stdout:osd.1: {} 2024-04-26T04:33:28.679 INFO:teuthology.orchestra.run.smithi012.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-26T04:33:28.713 INFO:teuthology.orchestra.run.smithi012.stdout:osd.2: {} 2024-04-26T04:33:28.713 INFO:teuthology.orchestra.run.smithi012.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-26T04:33:28.728 INFO:tasks.ceph.osd.8.smithi012.stderr:2024-04-26T04:33:28.721+0000 7f2884cf1640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25082) UID: 0 2024-04-26T04:33:28.734 INFO:teuthology.orchestra.run.smithi012.stdout:osd.3: {} 2024-04-26T04:33:28.734 INFO:teuthology.orchestra.run.smithi012.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-26T04:33:28.744 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:28.744 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:28.744 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:33:28.745 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:33:28.745 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:28.752 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:33:28.753 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:33:28.754 INFO:teuthology.orchestra.run.smithi012.stdout:osd.4: {} 2024-04-26T04:33:28.754 INFO:teuthology.orchestra.run.smithi012.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-26T04:33:28.758 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-26T04:33:28.770 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-04-26T04:33:28.395783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":89735712,"num_objects":59,"num_object_clones":14,"num_object_copies":403,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":5,"num_read":654,"num_read_kb":275953,"num_write":1038,"num_write_kb":135076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":33776160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1356,"ondisk_log_size":1356,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":14,"num_per_pool_osds":14,"num_per_pool_omap_osds":14,"kb":1321205760,"kb_used":612736,"kb_used_data":176216,"kb_used_omap":21,"kb_used_meta":436330,"kb_avail":1320593024,"statfs":{"total":1352914698240,"available":1352287256576,"internally_reserved":0,"allocated":180445184,"data_stored":207912847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22326,"internal_metadata":446802122},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,6,8,12,3,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7159808,"num_objects":-5,"num_object_clones":-3,"num_object_copies":-35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":2,"num_read":21,"num_read_kb":28310,"num_write":73,"num_write_kb":336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":28164096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.029071"},"pg_stats":[{"pgid":"3.f","version":"88'139","reported_seq":460,"reported_epoch":89,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T04:33:24.426702+0000","last_change":"2024-04-26T04:33:22.229805+0000","last_active":"2024-04-26T04:33:24.426702+0000","last_peered":"2024-04-26T04:33:24.426702+0000","last_clean":"2024-04-26T04:33:18.913521+0000","last_became_active":"2024-04-26T04:33:21.404150+0000","last_became_peered":"2024-04-26T04:33:21.404150+0000","last_unstale":"2024-04-26T04:33:24.426702+0000","last_undegraded":"2024-04-26T04:33:21.195630+0000","last_fullsized":"2024-04-26T04:33:21.193197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":3,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.60452043499999997,"stat_sum":{"num_bytes":9158656,"num_objects":10,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":60,"num_read_kb":8850,"num_write":123,"num_write_kb":13568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":6930432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,14,13,8,3,0],"acting":[11,1,14,13,8,2147483647,0],"avail_no_missing":["11(0)","0(6)","1(1)","8(4)","10(5)","13(3)","14(2)"],"object_location_counts":[{"shards":"0(6),1(1),8(4),11(0),13(3),14(2)","objects":9}],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"10","length":"4"},{"start":"15","length":"1"},{"start":"18","length":"1"}]},{"pgid":"3.e","version":"89'113","reported_seq":389,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T04:33:25.387199+0000","last_change":"2024-04-26T04:33:25.286123+0000","last_active":"2024-04-26T04:33:25.387199+0000","last_peered":"2024-04-26T04:33:25.387199+0000","last_clean":"2024-04-26T04:33:25.387199+0000","last_became_active":"2024-04-26T04:32:23.387938+0000","last_became_peered":"2024-04-26T04:32:23.387938+0000","last_unstale":"2024-04-26T04:33:25.387199+0000","last_undegraded":"2024-04-26T04:33:25.387199+0000","last_fullsized":"2024-04-26T04:33:25.387199+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:45.784981+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.81269671700000001,"stat_sum":{"num_bytes":12288000,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24206,"num_write":50,"num_write_kb":14880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,9,1,11,8,6,12],"acting":[13,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.d","version":"82'69","reported_seq":381,"reported_epoch":89,"state":"active+undersized+remapped","last_fresh":"2024-04-26T04:33:24.710754+0000","last_change":"2024-04-26T04:33:24.711662+0000","last_active":"2024-04-26T04:33:24.710754+0000","last_peered":"2024-04-26T04:33:24.710754+0000","last_clean":"2024-04-26T04:33:18.281597+0000","last_became_active":"2024-04-26T04:33:21.502298+0000","last_became_peered":"2024-04-26T04:33:21.502298+0000","last_unstale":"2024-04-26T04:33:24.710754+0000","last_undegraded":"2024-04-26T04:33:24.710754+0000","last_fullsized":"2024-04-26T04:33:21.291963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10101510399999999,"stat_sum":{"num_bytes":3538944,"num_objects":3,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":16825,"num_write":68,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":3538944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,9,0,8,4],"acting":[7,3,14,2147483647,0,8,4],"avail_no_missing":["7(0)","0(4)","3(1)","4(6)","8(5)","9(3)","10(3)","14(2)"],"object_location_counts":[{"shards":"0(4),3(1),4(6),7(0),8(5),14(2)","objects":3}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"10","length":"6"},{"start":"18","length":"1"}]},{"pgid":"3.c","version":"0'0","reported_seq":280,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T04:33:24.477952+0000","last_change":"2024-04-26T04:33:24.578755+0000","last_active":"2024-04-26T04:33:24.477952+0000","last_peered":"2024-04-26T04:33:24.477952+0000","last_clean":"2024-04-26T04:33:24.477952+0000","last_became_active":"2024-04-26T04:33:20.491311+0000","last_became_peered":"2024-04-26T04:33:20.491311+0000","last_unstale":"2024-04-26T04:33:24.477952+0000","last_undegraded":"2024-04-26T04:33:24.477952+0000","last_fullsized":"2024-04-26T04:33:24.477952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:33.949495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.102672491,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,9,6,12,5],"acting":[11,8,0,9,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.b","version":"89'101","reported_seq":412,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T04:33:27.535849+0000","last_change":"2024-04-26T04:33:25.004998+0000","last_active":"2024-04-26T04:33:27.535849+0000","last_peered":"2024-04-26T04:33:27.535849+0000","last_clean":"2024-04-26T04:33:27.535849+0000","last_became_active":"2024-04-26T04:32:23.220443+0000","last_became_peered":"2024-04-26T04:32:23.220443+0000","last_unstale":"2024-04-26T04:33:27.535849+0000","last_undegraded":"2024-04-26T04:33:27.535849+0000","last_fullsized":"2024-04-26T04:33:27.535849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:46.954175+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.55346476,"stat_sum":{"num_bytes":5423104,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":69,"num_read_kb":36654,"num_write":57,"num_write_kb":11024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"},{"start":"16","length":"2"}]},{"pgid":"3.a","version":"89'66","reported_seq":421,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T04:33:24.813320+0000","last_change":"2024-04-26T04:33:24.786685+0000","last_active":"2024-04-26T04:33:24.813320+0000","last_peered":"2024-04-26T04:33:24.813320+0000","last_clean":"2024-04-26T04:33:24.813320+0000","last_became_active":"2024-04-26T04:33:16.588782+0000","last_became_peered":"2024-04-26T04:33:16.588782+0000","last_unstale":"2024-04-26T04:33:24.813320+0000","last_undegraded":"2024-04-26T04:33:24.813320+0000","last_fullsized":"2024-04-26T04:33:24.813320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:50.036231+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.30923474200000001,"stat_sum":{"num_bytes":3489792,"num_objects":4,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":31,"num_read_kb":24969,"num_write":24,"num_write_kb":5216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":3309568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,13,6,9,8],"acting":[3,1,7,13,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.9","version":"82'16","reported_seq":321,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T04:33:25.472698+0000","last_change":"2024-04-26T04:33:24.897095+0000","last_active":"2024-04-26T04:33:25.472698+0000","last_peered":"2024-04-26T04:33:25.472698+0000","last_clean":"2024-04-26T04:33:25.472698+0000","last_became_active":"2024-04-26T04:33:24.680939+0000","last_became_peered":"2024-04-26T04:33:24.680939+0000","last_unstale":"2024-04-26T04:33:25.472698+0000","last_undegraded":"2024-04-26T04:33:25.472698+0000","last_fullsized":"2024-04-26T04:33:25.472698+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:41.427684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.103369248,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":4132,"num_write":23,"num_write_kb":448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,4],"acting":[14,1,0,3,15,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.8","version":"89'146","reported_seq":434,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T04:33:25.425323+0000","last_change":"2024-04-26T04:33:25.317185+0000","last_active":"2024-04-26T04:33:25.425323+0000","last_peered":"2024-04-26T04:33:25.425323+0000","last_clean":"2024-04-26T04:33:25.425323+0000","last_became_active":"2024-04-26T04:32:23.487565+0000","last_became_peered":"2024-04-26T04:32:23.487565+0000","last_unstale":"2024-04-26T04:33:25.425323+0000","last_undegraded":"2024-04-26T04:33:25.425323+0000","last_fullsized":"2024-04-26T04:33:25.425323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:48.554731+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.86665977900000002,"stat_sum":{"num_bytes":13008896,"num_objects":7,"num_object_clones":1,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":55,"num_read_kb":34917,"num_write":116,"num_write_kb":16336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,12,1,14,15,6],"acting":[5,0,12,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"1.0","version":"32'157","reported_seq":299,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T04:33:24.476453+0000","last_change":"2024-04-26T04:32:56.378977+0000","last_active":"2024-04-26T04:33:24.476453+0000","last_peered":"2024-04-26T04:33:24.476453+0000","last_clean":"2024-04-26T04:33:24.476453+0000","last_became_active":"2024-04-26T04:32:56.378173+0000","last_became_peered":"2024-04-26T04:32:56.378173+0000","last_unstale":"2024-04-26T04:33:24.476453+0000","last_undegraded":"2024-04-26T04:33:24.476453+0000","last_fullsized":"2024-04-26T04:33:24.476453+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:32:08.365921+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:25.422212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,13],"acting":[3,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":146,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T04:33:25.392627+0000","last_change":"2024-04-26T04:33:19.895062+0000","last_active":"2024-04-26T04:33:25.392627+0000","last_peered":"2024-04-26T04:33:25.392627+0000","last_clean":"2024-04-26T04:33:25.392627+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:33:25.392627+0000","last_undegraded":"2024-04-26T04:33:25.392627+0000","last_fullsized":"2024-04-26T04:33:25.392627+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:19.894942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:19.894942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.035115+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"80'18","reported_seq":233,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T04:33:27.521409+0000","last_change":"2024-04-26T04:33:24.978470+0000","last_active":"2024-04-26T04:33:27.521409+0000","last_peered":"2024-04-26T04:33:27.521409+0000","last_clean":"2024-04-26T04:33:27.521409+0000","last_became_active":"2024-04-26T04:33:24.674496+0000","last_became_peered":"2024-04-26T04:33:24.674496+0000","last_unstale":"2024-04-26T04:33:27.521409+0000","last_undegraded":"2024-04-26T04:33:27.521409+0000","last_fullsized":"2024-04-26T04:33:27.521409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:33.098450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20325084199999999,"stat_sum":{"num_bytes":2736128,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":4231,"num_write":14,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,0,3,6,15,1,11],"acting":[13,0,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"1"},{"start":"16","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":143,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T04:33:24.476425+0000","last_change":"2024-04-26T04:33:19.350938+0000","last_active":"2024-04-26T04:33:24.476425+0000","last_peered":"2024-04-26T04:33:24.476425+0000","last_clean":"2024-04-26T04:33:24.476425+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:33:24.476425+0000","last_undegraded":"2024-04-26T04:33:24.476425+0000","last_fullsized":"2024-04-26T04:33:24.476425+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:19.350538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:19.350538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:36.961751+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"92'105","reported_seq":425,"reported_epoch":92,"state":"active+undersized+remapped","last_fresh":"2024-04-26T04:33:28.046696+0000","last_change":"2024-04-26T04:33:27.747117+0000","last_active":"2024-04-26T04:33:28.046696+0000","last_peered":"2024-04-26T04:33:28.046696+0000","last_clean":"2024-04-26T04:33:18.402876+0000","last_became_active":"2024-04-26T04:33:21.595704+0000","last_became_peered":"2024-04-26T04:33:21.595704+0000","last_unstale":"2024-04-26T04:33:28.046696+0000","last_undegraded":"2024-04-26T04:33:28.046696+0000","last_fullsized":"2024-04-26T04:33:21.193390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10194238,"stat_sum":{"num_bytes":13041664,"num_objects":8,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":43,"num_read_kb":44564,"num_write":78,"num_write_kb":15520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":19529728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,15,12,13],"acting":[4,11,9,5,2147483647,12,13],"avail_no_missing":["4(0)","5(3)","9(2)","10(4)","11(1)","12(5)","13(6)","15(4)"],"object_location_counts":[{"shards":"4(0),5(3),9(2),11(1),12(5),13(6)","objects":8}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":69,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T04:33:25.394130+0000","last_change":"2024-04-26T04:32:56.277333+0000","last_active":"2024-04-26T04:33:25.394130+0000","last_peered":"2024-04-26T04:33:25.394130+0000","last_clean":"2024-04-26T04:33:25.394130+0000","last_became_active":"2024-04-26T04:32:56.276685+0000","last_became_peered":"2024-04-26T04:32:56.276685+0000","last_unstale":"2024-04-26T04:33:25.394130+0000","last_undegraded":"2024-04-26T04:33:25.394130+0000","last_fullsized":"2024-04-26T04:33:25.394130+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.299653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,13],"acting":[5,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"89'21","reported_seq":303,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T04:33:25.372474+0000","last_change":"2024-04-26T04:33:24.757945+0000","last_active":"2024-04-26T04:33:25.372474+0000","last_peered":"2024-04-26T04:33:25.372474+0000","last_clean":"2024-04-26T04:33:25.372474+0000","last_became_active":"2024-04-26T04:32:23.386975+0000","last_became_peered":"2024-04-26T04:32:23.386975+0000","last_unstale":"2024-04-26T04:33:25.372474+0000","last_undegraded":"2024-04-26T04:33:25.372474+0000","last_fullsized":"2024-04-26T04:33:25.372474+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:42.363133+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.284215887,"stat_sum":{"num_bytes":1654784,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":9700,"num_write":30,"num_write_kb":1744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,11,15,1,5,6,4],"acting":[14,11,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"2.2","version":"0'0","reported_seq":135,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T04:33:24.577946+0000","last_change":"2024-04-26T04:32:12.798464+0000","last_active":"2024-04-26T04:33:24.577946+0000","last_peered":"2024-04-26T04:33:24.577946+0000","last_clean":"2024-04-26T04:33:24.577946+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:33:24.577946+0000","last_undegraded":"2024-04-26T04:33:24.577946+0000","last_fullsized":"2024-04-26T04:33:24.577946+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:35.684262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":273,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T04:33:25.473296+0000","last_change":"2024-04-26T04:33:24.476667+0000","last_active":"2024-04-26T04:33:25.473296+0000","last_peered":"2024-04-26T04:33:25.473296+0000","last_clean":"2024-04-26T04:33:25.473296+0000","last_became_active":"2024-04-26T04:32:23.387136+0000","last_became_peered":"2024-04-26T04:32:23.387136+0000","last_unstale":"2024-04-26T04:33:25.473296+0000","last_undegraded":"2024-04-26T04:33:25.473296+0000","last_fullsized":"2024-04-26T04:33:25.473296+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:46.920410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.100755753,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,6,5,3],"acting":[13,7,4,15,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"10","length":"9"}]},{"pgid":"3.5","version":"89'133","reported_seq":423,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T04:33:27.483598+0000","last_change":"2024-04-26T04:33:25.212668+0000","last_active":"2024-04-26T04:33:27.483598+0000","last_peered":"2024-04-26T04:33:27.483598+0000","last_clean":"2024-04-26T04:33:27.483598+0000","last_became_active":"2024-04-26T04:32:23.295127+0000","last_became_peered":"2024-04-26T04:32:23.295127+0000","last_unstale":"2024-04-26T04:33:27.483598+0000","last_undegraded":"2024-04-26T04:33:27.483598+0000","last_fullsized":"2024-04-26T04:33:27.483598+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:50.361566+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.73486344699999995,"stat_sum":{"num_bytes":4833280,"num_objects":5,"num_object_clones":1,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":45,"num_read_kb":17105,"num_write":86,"num_write_kb":10048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"14","length":"1"},{"start":"16","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T04:33:27.535800+0000","last_change":"2024-04-26T04:32:13.108823+0000","last_active":"2024-04-26T04:33:27.535800+0000","last_peered":"2024-04-26T04:33:27.535800+0000","last_clean":"2024-04-26T04:33:27.535800+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:33:27.535800+0000","last_undegraded":"2024-04-26T04:33:27.535800+0000","last_fullsized":"2024-04-26T04:33:27.535800+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"90'119","reported_seq":407,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T04:33:27.489575+0000","last_change":"2024-04-26T04:33:25.663152+0000","last_active":"2024-04-26T04:33:27.489575+0000","last_peered":"2024-04-26T04:33:27.489575+0000","last_clean":"2024-04-26T04:33:27.489575+0000","last_became_active":"2024-04-26T04:32:23.290645+0000","last_became_peered":"2024-04-26T04:32:23.290645+0000","last_unstale":"2024-04-26T04:33:27.489575+0000","last_undegraded":"2024-04-26T04:33:27.489575+0000","last_fullsized":"2024-04-26T04:33:27.489575+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:33.273678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20548291499999999,"stat_sum":{"num_bytes":4816896,"num_objects":6,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":12398,"num_write":62,"num_write_kb":11040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,4,1,0,14,13],"acting":[8,9,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"14","length":"1"},{"start":"16","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":135,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T04:33:24.477767+0000","last_change":"2024-04-26T04:32:12.504400+0000","last_active":"2024-04-26T04:33:24.477767+0000","last_peered":"2024-04-26T04:33:24.477767+0000","last_clean":"2024-04-26T04:33:24.477767+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:33:24.477767+0000","last_undegraded":"2024-04-26T04:33:24.477767+0000","last_fullsized":"2024-04-26T04:33:24.477767+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:37.381954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"89'89","reported_seq":390,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T04:33:27.435817+0000","last_change":"2024-04-26T04:33:24.805797+0000","last_active":"2024-04-26T04:33:27.435817+0000","last_peered":"2024-04-26T04:33:27.435817+0000","last_clean":"2024-04-26T04:33:27.435817+0000","last_became_active":"2024-04-26T04:32:23.477983+0000","last_became_peered":"2024-04-26T04:32:23.477983+0000","last_unstale":"2024-04-26T04:33:27.435817+0000","last_undegraded":"2024-04-26T04:33:27.435817+0000","last_fullsized":"2024-04-26T04:33:27.435817+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.35409425700000002,"stat_sum":{"num_bytes":4997120,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":27144,"num_write":32,"num_write_kb":11056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"14","length":"1"},{"start":"16","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":140,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T04:33:27.489992+0000","last_change":"2024-04-26T04:32:12.697838+0000","last_active":"2024-04-26T04:33:27.489992+0000","last_peered":"2024-04-26T04:33:27.489992+0000","last_clean":"2024-04-26T04:33:27.489992+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:33:27.489992+0000","last_undegraded":"2024-04-26T04:33:27.489992+0000","last_fullsized":"2024-04-26T04:33:27.489992+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:40.961816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"82'64","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T04:33:27.521559+0000","last_change":"2024-04-26T04:33:24.372943+0000","last_active":"2024-04-26T04:33:27.521559+0000","last_peered":"2024-04-26T04:33:27.521559+0000","last_clean":"2024-04-26T04:33:27.521559+0000","last_became_active":"2024-04-26T04:32:23.383105+0000","last_became_peered":"2024-04-26T04:32:23.383105+0000","last_unstale":"2024-04-26T04:33:27.521559+0000","last_undegraded":"2024-04-26T04:33:27.521559+0000","last_fullsized":"2024-04-26T04:33:27.521559+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:22.154164+0000","last_clean_scrub_stamp":"2024-04-26T04:32:22.154164+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:47.470067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095213800000000005,"stat_sum":{"num_bytes":10158080,"num_objects":2,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":10140,"num_write":72,"num_write_kb":9920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"1"},{"start":"16","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":140,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T04:33:27.483614+0000","last_change":"2024-04-26T04:32:12.803651+0000","last_active":"2024-04-26T04:33:27.483614+0000","last_peered":"2024-04-26T04:33:27.483614+0000","last_clean":"2024-04-26T04:33:27.483614+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:33:27.483614+0000","last_undegraded":"2024-04-26T04:33:27.483614+0000","last_fullsized":"2024-04-26T04:33:27.483614+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:32:11.466714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":89145344,"num_objects":57,"num_object_clones":14,"num_object_copies":399,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":5,"num_read":518,"num_read_kb":275835,"num_write":835,"num_write_kb":131616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":33308672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":162107392,"data_stored":192745472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1199,"ondisk_log_size":1199,"up":112,"acting":112,"num_store_stats":15},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1536000,"data_stored":1525344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247827,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46544,"kb_used_data":15364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325296,"statfs":{"total":96636764160,"available":96589103104,"internally_reserved":0,"allocated":15732736,"data_stored":20933796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247828,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40876,"kb_used_data":9696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330964,"statfs":{"total":96636764160,"available":96594907136,"internally_reserved":0,"allocated":9928704,"data_stored":10980516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,2,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247827,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48396,"kb_used_data":17216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323444,"statfs":{"total":96636764160,"available":96587206656,"internally_reserved":0,"allocated":17629184,"data_stored":22814404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313235,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46704,"kb_used_data":15524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325136,"statfs":{"total":96636764160,"available":96588939264,"internally_reserved":0,"allocated":15896576,"data_stored":16075898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,5,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313235,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41836,"kb_used_data":10648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330004,"statfs":{"total":96636764160,"available":96593924096,"internally_reserved":0,"allocated":10903552,"data_stored":13995130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,5,6,7,8,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313235,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42904,"kb_used_data":11728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328936,"statfs":{"total":96636764160,"available":96592830464,"internally_reserved":0,"allocated":12009472,"data_stored":12700794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,2,7,3,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":0,"seq":90194313235,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313235,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40976,"kb_used_data":13956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330864,"statfs":{"total":96636764160,"available":96594804736,"internally_reserved":0,"allocated":14290944,"data_stored":15021093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":0,"seq":98784247828,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280530,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":5652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":5787648,"data_stored":5854546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280530,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47896,"kb_used_data":12560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94323944,"statfs":{"total":96636764160,"available":96587718656,"internally_reserved":0,"allocated":12861440,"data_stored":16507698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247827,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47780,"kb_used_data":16596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324060,"statfs":{"total":96636764160,"available":96587837440,"internally_reserved":0,"allocated":16994304,"data_stored":17998969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247828,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40576,"kb_used_data":9396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331264,"statfs":{"total":96636764160,"available":96595214336,"internally_reserved":0,"allocated":9621504,"data_stored":11295908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247826,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46056,"kb_used_data":14876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325784,"statfs":{"total":96636764160,"available":96589602816,"internally_reserved":0,"allocated":15233024,"data_stored":17652900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280530,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43896,"kb_used_data":8560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94327944,"statfs":{"total":96636764160,"available":96591814656,"internally_reserved":0,"allocated":8765440,"data_stored":8766258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,2,1,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247828,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45628,"kb_used_data":14444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326212,"statfs":{"total":96636764160,"available":96590041088,"internally_reserved":0,"allocated":14790656,"data_stored":17314937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":467488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14569472,"data_stored":15302656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15683584,"data_stored":17235968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4878336,"data_stored":5550080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9576448,"data_stored":13221888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8175616,"data_stored":11087872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12107776,"data_stored":12939264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7532544,"data_stored":8421376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":10682368,"data_stored":11927552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":14438400,"data_stored":18755584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":3244032,"data_stored":3244032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":11583488,"data_stored":15216640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":13479936,"data_stored":16551936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":15130624,"data_stored":19435520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":8044544,"data_stored":8765440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":12980224,"data_stored":15089664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:33:28.772 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-04-26T04:33:28.772 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-04-26T04:33:28.773 INFO:teuthology.orchestra.run.smithi012.stdout:osd.5: {} 2024-04-26T04:33:28.773 INFO:teuthology.orchestra.run.smithi012.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-26T04:33:28.792 INFO:teuthology.orchestra.run.smithi012.stdout:osd.6: {} 2024-04-26T04:33:28.792 INFO:teuthology.orchestra.run.smithi012.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-26T04:33:28.828 INFO:teuthology.orchestra.run.smithi012.stdout:osd.7: {} 2024-04-26T04:33:28.829 INFO:teuthology.orchestra.run.smithi012.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-26T04:33:28.829 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-04-26T04:33:28.828+0000 7fac9be4b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25072) UID: 0 2024-04-26T04:33:28.857 INFO:teuthology.orchestra.run.smithi012.stdout:osd.8: {} 2024-04-26T04:33:28.858 INFO:teuthology.orchestra.run.smithi012.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-26T04:33:28.876 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:28.877 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:33:28.877 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:33:28.877 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:33:28.877 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.877 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:42 1.0 e26/23 1:30306672:devicehealth::main.db.0000000000000000:head v 26'28, mlcod=26'26)", 2024-04-26T04:33:28.877 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:33:28.877 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 75.656362051000002, 2024-04-26T04:33:28.877 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201587136, 2024-04-26T04:33:28.877 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.877 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.877 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.877 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.877 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.878 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.878 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:33:28.878 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.878 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.878 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.878 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.878 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220674+0000", 2024-04-26T04:33:28.878 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.878 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.879 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.879 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220676+0000", 2024-04-26T04:33:28.879 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.843e-06 2024-04-26T04:33:28.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.879 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.879 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220687+0000", 2024-04-26T04:33:28.879 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1727e-05 2024-04-26T04:33:28.879 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.879 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.879 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.220715+0000", 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8033000000000001e-05 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.320857+0000", 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014171400000001 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.421143+0000", 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100285902 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.880 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.421175+0000", 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2413999999999998e-05 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.422168+0000", 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0009931269999999999 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.422261+0000", 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2375999999999998e-05 2024-04-26T04:33:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.882 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.882 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.882 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.882 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.882 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:55 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'33, mlcod=27'29)", 2024-04-26T04:33:28.882 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:33:28.882 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 75.332972187999999, 2024-04-26T04:33:28.882 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.169473288, 2024-04-26T04:33:28.882 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.882 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.882 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.882 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.882 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544064+0000", 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544065+0000", 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.77e-06 2024-04-26T04:33:28.883 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.884 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.884 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.884 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544195+0000", 2024-04-26T04:33:28.884 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012971599999999999 2024-04-26T04:33:28.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.884 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.884 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.884 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.544219+0000", 2024-04-26T04:33:28.884 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3476e-05 2024-04-26T04:33:28.884 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.884 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.884 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.885 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.644316+0000", 2024-04-26T04:33:28.885 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100097189 2024-04-26T04:33:28.885 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.885 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.885 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.885 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.711737+0000", 2024-04-26T04:33:28.885 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.067420846000000006 2024-04-26T04:33:28.886 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.886 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.886 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.886 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.711770+0000", 2024-04-26T04:33:28.886 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2959000000000001e-05 2024-04-26T04:33:28.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.887 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.887 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.887 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.713428+0000", 2024-04-26T04:33:28.887 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016584130000000001 2024-04-26T04:33:28.887 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.887 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.887 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.887 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:13.713537+0000", 2024-04-26T04:33:28.887 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010891900000000001 2024-04-26T04:33:28.887 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.888 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.888 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.888 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.888 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.888 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:108 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'70, mlcod=27'64)", 2024-04-26T04:33:28.888 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:33:28.888 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 72.068578024999994, 2024-04-26T04:33:28.888 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30260933299999998, 2024-04-26T04:33:28.888 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.888 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.888 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.889 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808458+0000", 2024-04-26T04:33:28.890 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.37e-07 2024-04-26T04:33:28.890 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.890 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.890 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.890 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808464+0000", 2024-04-26T04:33:28.890 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8599999999999998e-06 2024-04-26T04:33:28.890 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.890 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.890 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.890 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.808474+0000", 2024-04-26T04:33:28.890 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.9830000000000008e-06 2024-04-26T04:33:28.890 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.890 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908648+0000", 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10017390299999999 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908808+0000", 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000159576 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908845+0000", 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7026000000000001e-05 2024-04-26T04:33:28.891 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.892 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.892 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.892 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110983+0000", 2024-04-26T04:33:28.892 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20213835199999999 2024-04-26T04:33:28.892 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.892 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.892 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.892 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111067+0000", 2024-04-26T04:33:28.892 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.3996e-05 2024-04-26T04:33:28.892 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.892 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.892 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.892 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.893 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.893 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:109 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'71, mlcod=27'64)", 2024-04-26T04:33:28.893 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:33:28.893 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 71.968343301999994, 2024-04-26T04:33:28.893 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202472819, 2024-04-26T04:33:28.893 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.893 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.893 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.893 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.893 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.893 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.893 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:33:28.893 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.894 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.894 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.894 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.894 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908692+0000", 2024-04-26T04:33:28.894 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.894 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.894 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.894 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.894 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908694+0000", 2024-04-26T04:33:28.894 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.144e-06 2024-04-26T04:33:28.894 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.894 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.894 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.895 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908751+0000", 2024-04-26T04:33:28.895 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7268999999999997e-05 2024-04-26T04:33:28.895 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.895 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.895 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.895 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908769+0000", 2024-04-26T04:33:28.895 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8363e-05 2024-04-26T04:33:28.895 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.895 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.896 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.896 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908780+0000", 2024-04-26T04:33:28.896 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.096e-05 2024-04-26T04:33:28.896 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.896 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.896 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.896 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.909341+0000", 2024-04-26T04:33:28.897 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00056057500000000001 2024-04-26T04:33:28.897 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.897 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.897 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.897 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.909371+0000", 2024-04-26T04:33:28.898 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0357000000000001e-05 2024-04-26T04:33:28.898 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.898 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.898 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.898 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111074+0000", 2024-04-26T04:33:28.898 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20170292500000001 2024-04-26T04:33:28.899 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.899 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.899 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.899 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111165+0000", 2024-04-26T04:33:28.899 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.1225999999999997e-05 2024-04-26T04:33:28.899 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.899 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.899 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:110 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'72, mlcod=27'64)", 2024-04-26T04:33:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:33:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 71.968208410000003, 2024-04-26T04:33:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20385624999999999, 2024-04-26T04:33:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.900 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908827+0000", 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908828+0000", 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9400000000000005e-07 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.901 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.902 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908836+0000", 2024-04-26T04:33:28.902 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1929999999999993e-06 2024-04-26T04:33:28.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.902 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.902 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.902 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908847+0000", 2024-04-26T04:33:28.902 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0703000000000001e-05 2024-04-26T04:33:28.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.902 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.902 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.902 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908855+0000", 2024-04-26T04:33:28.902 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7829999999999999e-06 2024-04-26T04:33:28.902 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009054+0000", 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10019966900000001 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009089+0000", 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4433e-05 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112582+0000", 2024-04-26T04:33:28.903 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103492973 2024-04-26T04:33:28.904 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.904 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.904 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.904 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112684+0000", 2024-04-26T04:33:28.904 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000101802 2024-04-26T04:33:28.904 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.904 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.904 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.904 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.904 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.904 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:111 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'73, mlcod=27'64)", 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 71.96816948, 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.205422306, 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.905 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.906 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908866+0000", 2024-04-26T04:33:28.906 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.906 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.906 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.906 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908867+0000", 2024-04-26T04:33:28.906 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.03e-07 2024-04-26T04:33:28.906 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.907 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.907 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.907 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908871+0000", 2024-04-26T04:33:28.907 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5800000000000002e-06 2024-04-26T04:33:28.907 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.907 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.907 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.907 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:16.908883+0000", 2024-04-26T04:33:28.908 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1507e-05 2024-04-26T04:33:28.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.908 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.908 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.908 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009022+0000", 2024-04-26T04:33:28.908 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100139593 2024-04-26T04:33:28.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.908 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.908 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.908 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.109703+0000", 2024-04-26T04:33:28.908 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100680741 2024-04-26T04:33:28.908 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.908 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.909 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.109738+0000", 2024-04-26T04:33:28.909 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5296000000000002e-05 2024-04-26T04:33:28.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.909 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114207+0000", 2024-04-26T04:33:28.909 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0044682239999999998 2024-04-26T04:33:28.909 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.909 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.909 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.909 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114289+0000", 2024-04-26T04:33:28.910 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2062000000000001e-05 2024-04-26T04:33:28.910 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.910 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.910 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.910 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.910 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.911 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:112 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'74, mlcod=27'64)", 2024-04-26T04:33:28.911 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:33:28.911 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 71.867975357999995, 2024-04-26T04:33:28.911 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105317487, 2024-04-26T04:33:28.911 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.912 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.912 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.912 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.912 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.912 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.912 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:33:28.913 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.913 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.913 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.913 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.913 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009060+0000", 2024-04-26T04:33:28.913 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.914 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.914 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.914 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.914 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009061+0000", 2024-04-26T04:33:28.914 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.057e-06 2024-04-26T04:33:28.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.915 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.915 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.915 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009128+0000", 2024-04-26T04:33:28.915 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6859999999999993e-05 2024-04-26T04:33:28.915 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.916 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.916 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.916 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009147+0000", 2024-04-26T04:33:28.916 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8196e-05 2024-04-26T04:33:28.916 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.916 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.917 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.917 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009157+0000", 2024-04-26T04:33:28.917 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0743e-05 2024-04-26T04:33:28.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.918 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.918 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110254+0000", 2024-04-26T04:33:28.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101096941 2024-04-26T04:33:28.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.918 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.918 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.918 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.110311+0000", 2024-04-26T04:33:28.918 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6718999999999997e-05 2024-04-26T04:33:28.918 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114296+0000", 2024-04-26T04:33:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0039847709999999998 2024-04-26T04:33:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114378+0000", 2024-04-26T04:33:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2200000000000006e-05 2024-04-26T04:33:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.919 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:114 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'76, mlcod=27'64)", 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 71.867826109000006, 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105329463, 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.920 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.921 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.921 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.921 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:33:28.921 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.921 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.921 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.921 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009210+0000", 2024-04-26T04:33:28.921 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.53e-07 2024-04-26T04:33:28.921 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.921 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.921 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.921 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009215+0000", 2024-04-26T04:33:28.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7099999999999998e-06 2024-04-26T04:33:28.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.922 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.922 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009223+0000", 2024-04-26T04:33:28.922 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.5599999999999994e-06 2024-04-26T04:33:28.922 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.922 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.922 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009229+0000", 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0800000000000002e-06 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111865+0000", 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102636071 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.111941+0000", 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5615e-05 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.923 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114468+0000", 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002526875 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114539+0000", 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.1198999999999994e-05 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:115 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'77, mlcod=27'64)", 2024-04-26T04:33:28.924 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 71.867798235999999, 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105390922, 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.925 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:33:28.926 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.926 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.926 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.926 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009238+0000", 2024-04-26T04:33:28.926 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.34e-07 2024-04-26T04:33:28.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.926 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.926 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.926 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009293+0000", 2024-04-26T04:33:28.926 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4954999999999998e-05 2024-04-26T04:33:28.926 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.927 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.927 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009304+0000", 2024-04-26T04:33:28.927 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1263999999999999e-05 2024-04-26T04:33:28.927 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.927 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.927 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.927 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009311+0000", 2024-04-26T04:33:28.927 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6630000000000004e-06 2024-04-26T04:33:28.929 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.929 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.930 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.930 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112802+0000", 2024-04-26T04:33:28.930 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10349130600000001 2024-04-26T04:33:28.930 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.930 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.930 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.930 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.112843+0000", 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0472999999999997e-05 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114547+0000", 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001704552 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114628+0000", 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1374999999999999e-05 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.931 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.932 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.932 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.932 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:116 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'78, mlcod=27'64)", 2024-04-26T04:33:28.932 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:33:28.932 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 71.867714982999999, 2024-04-26T04:33:28.932 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105877241, 2024-04-26T04:33:28.932 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.932 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.932 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.932 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.932 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.932 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.932 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:33:28.933 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.933 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.933 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.933 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.933 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:33:28.933 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.933 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.933 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.933 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.933 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009321+0000", 2024-04-26T04:33:28.933 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.46e-07 2024-04-26T04:33:28.934 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.934 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.934 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.934 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009326+0000", 2024-04-26T04:33:28.934 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5040000000000004e-06 2024-04-26T04:33:28.934 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.934 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.934 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.934 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009334+0000", 2024-04-26T04:33:28.934 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1929999999999993e-06 2024-04-26T04:33:28.934 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.934 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.934 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009340+0000", 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.3729999999999998e-06 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.113595+0000", 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104254889 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.113649+0000", 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3887000000000001e-05 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.935 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115105+0000", 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014560969999999999 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115198+0000", 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2851999999999998e-05 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.936 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:117 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'79, mlcod=27'64)", 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 71.867687500000002, 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10639752, 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.937 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.938 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009348+0000", 2024-04-26T04:33:28.938 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.938 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.938 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.938 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.938 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009349+0000", 2024-04-26T04:33:28.938 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7300000000000002e-07 2024-04-26T04:33:28.938 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.938 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.938 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.938 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009353+0000", 2024-04-26T04:33:28.938 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5970000000000002e-06 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009361+0000", 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.0600000000000008e-06 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.009381+0000", 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9533000000000001e-05 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.939 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114709+0000", 2024-04-26T04:33:28.940 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105328753 2024-04-26T04:33:28.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.940 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.940 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.940 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.114745+0000", 2024-04-26T04:33:28.940 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5976000000000002e-05 2024-04-26T04:33:28.940 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.940 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.941 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.941 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115683+0000", 2024-04-26T04:33:28.941 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00093774899999999998 2024-04-26T04:33:28.941 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.941 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.941 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.941 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:17.115746+0000", 2024-04-26T04:33:28.941 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2578999999999998e-05 2024-04-26T04:33:28.941 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.941 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.942 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.942 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.942 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.942 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:155 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'109, mlcod=27'105)", 2024-04-26T04:33:28.942 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:33:28.942 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 70.037499726999997, 2024-04-26T04:33:28.942 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20562082700000001, 2024-04-26T04:33:28.942 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.942 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.942 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.942 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.943 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.943 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.943 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:33:28.943 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.943 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.943 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.943 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.943 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839536+0000", 2024-04-26T04:33:28.944 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.944 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.944 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.944 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.944 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839537+0000", 2024-04-26T04:33:28.944 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7400000000000003e-07 2024-04-26T04:33:28.944 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.944 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.944 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.944 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839638+0000", 2024-04-26T04:33:28.944 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000101365 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.839653+0000", 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4874e-05 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939797+0000", 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014414100000001 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.945 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.040092+0000", 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100295305 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.040151+0000", 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8295999999999999e-05 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045057+0000", 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0049061390000000003 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.946 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045157+0000", 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000100133 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:156 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'110, mlcod=27'105)", 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 69.937204249999994, 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10750937100000001, 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.947 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939832+0000", 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.948 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939833+0000", 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1030000000000001e-06 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939849+0000", 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6005999999999999e-05 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939866+0000", 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7513e-05 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.949 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.950 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939876+0000", 2024-04-26T04:33:28.950 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0154e-05 2024-04-26T04:33:28.950 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.950 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.950 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.950 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.041330+0000", 2024-04-26T04:33:28.950 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101453864 2024-04-26T04:33:28.950 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.950 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.950 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.950 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.041398+0000", 2024-04-26T04:33:28.951 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7617e-05 2024-04-26T04:33:28.951 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.951 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.951 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.951 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047245+0000", 2024-04-26T04:33:28.951 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058470140000000002 2024-04-26T04:33:28.951 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.951 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.951 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.951 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047341+0000", 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6100000000000005e-05 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:157 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'111, mlcod=27'105)", 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 69.937104110999996, 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107487887, 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.952 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.953 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.953 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.953 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:33:28.953 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.953 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.953 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.953 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.953 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:33:28.953 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.953 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.953 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.953 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.953 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939932+0000", 2024-04-26T04:33:28.954 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.2300000000000001e-07 2024-04-26T04:33:28.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.954 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.954 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.954 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939942+0000", 2024-04-26T04:33:28.954 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4369999999999996e-06 2024-04-26T04:33:28.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.954 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.954 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.954 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939952+0000", 2024-04-26T04:33:28.954 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.075e-05 2024-04-26T04:33:28.954 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939963+0000", 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0069e-05 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.042203+0000", 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102240215 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.042259+0000", 2024-04-26T04:33:28.955 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5955999999999998e-05 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047353+0000", 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0050939080000000003 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.047420+0000", 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6928999999999996e-05 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.956 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:158 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'112, mlcod=27'105)", 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 69.937061525000004, 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.209867842, 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.957 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.958 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.958 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939974+0000", 2024-04-26T04:33:28.958 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.958 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.958 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.958 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.939975+0000", 2024-04-26T04:33:28.958 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8599999999999999e-07 2024-04-26T04:33:28.958 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.958 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.958 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.958 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940084+0000", 2024-04-26T04:33:28.958 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000109746 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940099+0000", 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4392999999999999e-05 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940110+0000", 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1236999999999999e-05 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043002+0000", 2024-04-26T04:33:28.959 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102892262 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043054+0000", 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1922999999999997e-05 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149774+0000", 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10672018599999999 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.960 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149842+0000", 2024-04-26T04:33:28.961 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.7708999999999998e-05 2024-04-26T04:33:28.961 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.961 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.961 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.961 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.961 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.961 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:159 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'113, mlcod=27'105)", 2024-04-26T04:33:28.961 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:33:28.961 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 69.936912903000007, 2024-04-26T04:33:28.961 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20976125000000001, 2024-04-26T04:33:28.961 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.962 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.962 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.962 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.962 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.962 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.962 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:33:28.962 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.962 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.962 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.962 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.962 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:33:28.963 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.963 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.963 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.963 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.963 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940123+0000", 2024-04-26T04:33:28.963 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7000000000000005e-07 2024-04-26T04:33:28.963 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.963 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.963 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.963 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940131+0000", 2024-04-26T04:33:28.963 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6369999999999999e-06 2024-04-26T04:33:28.963 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.963 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940143+0000", 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1603000000000001e-05 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940165+0000", 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2042999999999999e-05 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043777+0000", 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103612145 2024-04-26T04:33:28.964 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.965 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.965 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.965 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.043807+0000", 2024-04-26T04:33:28.965 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0562999999999999e-05 2024-04-26T04:33:28.965 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.965 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.965 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.965 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149845+0000", 2024-04-26T04:33:28.965 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10603802900000001 2024-04-26T04:33:28.965 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.965 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.965 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.966 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149884+0000", 2024-04-26T04:33:28.966 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8659999999999999e-05 2024-04-26T04:33:28.966 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.966 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.966 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.966 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.966 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.966 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:160 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'114, mlcod=27'105)", 2024-04-26T04:33:28.966 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:33:28.966 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 69.936862396999999, 2024-04-26T04:33:28.966 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20974973299999999, 2024-04-26T04:33:28.966 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.966 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.967 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.967 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.967 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.967 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.967 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:33:28.967 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.967 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.967 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.967 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.967 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940173+0000", 2024-04-26T04:33:28.967 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.967 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.967 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940174+0000", 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8999999999999998e-07 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940183+0000", 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6970000000000005e-06 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940194+0000", 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0983e-05 2024-04-26T04:33:28.968 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.969 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.969 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.969 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940201+0000", 2024-04-26T04:33:28.969 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4400000000000002e-06 2024-04-26T04:33:28.969 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.969 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.969 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.969 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.044326+0000", 2024-04-26T04:33:28.969 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104124984 2024-04-26T04:33:28.969 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.969 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.969 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.969 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.044377+0000", 2024-04-26T04:33:28.970 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1249999999999999e-05 2024-04-26T04:33:28.970 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.970 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.970 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.970 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149886+0000", 2024-04-26T04:33:28.970 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10550875999999999 2024-04-26T04:33:28.970 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.970 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.970 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.970 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149923+0000", 2024-04-26T04:33:28.971 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.7329000000000001e-05 2024-04-26T04:33:28.971 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.971 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.971 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.971 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.971 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.971 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:161 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'115, mlcod=27'105)", 2024-04-26T04:33:28.971 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:33:28.972 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 69.936826453999998, 2024-04-26T04:33:28.972 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.209755314, 2024-04-26T04:33:28.972 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.972 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.972 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.972 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.972 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.973 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.973 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:33:28.973 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.973 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.973 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.973 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.973 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940209+0000", 2024-04-26T04:33:28.974 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.974 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.974 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.974 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.974 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940210+0000", 2024-04-26T04:33:28.974 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.67e-07 2024-04-26T04:33:28.974 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.975 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.975 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.975 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940330+0000", 2024-04-26T04:33:28.975 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012061900000000001 2024-04-26T04:33:28.975 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.975 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.976 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.976 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940344+0000", 2024-04-26T04:33:28.976 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.367e-05 2024-04-26T04:33:28.976 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.976 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.976 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.976 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.940355+0000", 2024-04-26T04:33:28.977 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.083e-05 2024-04-26T04:33:28.977 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.977 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.977 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.977 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045239+0000", 2024-04-26T04:33:28.977 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104883873 2024-04-26T04:33:28.977 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.978 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.978 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.978 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.045278+0000", 2024-04-26T04:33:28.978 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9904000000000003e-05 2024-04-26T04:33:28.978 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.978 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.978 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.979 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149925+0000", 2024-04-26T04:33:28.979 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104646221 2024-04-26T04:33:28.979 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.979 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.979 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.979 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149965+0000", 2024-04-26T04:33:28.979 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9929999999999999e-05 2024-04-26T04:33:28.979 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.980 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.980 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.980 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.980 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.980 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:162 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'116, mlcod=27'105)", 2024-04-26T04:33:28.980 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:33:28.981 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 69.933362721999998, 2024-04-26T04:33:28.981 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.206336451, 2024-04-26T04:33:28.981 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.981 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.981 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.981 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.981 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.982 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.982 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:33:28.982 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.982 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.982 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.982 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.983 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943673+0000", 2024-04-26T04:33:28.983 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.983 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.983 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.983 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.983 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943675+0000", 2024-04-26T04:33:28.983 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8640000000000001e-06 2024-04-26T04:33:28.984 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.984 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.984 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.984 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943692+0000", 2024-04-26T04:33:28.984 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7503000000000002e-05 2024-04-26T04:33:28.984 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.984 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.984 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.984 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943720+0000", 2024-04-26T04:33:28.984 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7569999999999999e-05 2024-04-26T04:33:28.984 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.984 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.985 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.985 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.943734+0000", 2024-04-26T04:33:28.985 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3596e-05 2024-04-26T04:33:28.985 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.985 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.985 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.985 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.046517+0000", 2024-04-26T04:33:28.985 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102783897 2024-04-26T04:33:28.985 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.985 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.985 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.985 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.046569+0000", 2024-04-26T04:33:28.985 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1706000000000003e-05 2024-04-26T04:33:28.986 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.986 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.986 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.986 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.149968+0000", 2024-04-26T04:33:28.986 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10339902199999999 2024-04-26T04:33:28.986 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.986 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.986 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.986 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150009+0000", 2024-04-26T04:33:28.986 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1292999999999999e-05 2024-04-26T04:33:28.986 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.986 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.986 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.987 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.987 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.987 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_repop(client.4263.0:163 1.0 e27/23 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 27'117, mlcod=27'105)", 2024-04-26T04:33:28.987 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:33:28.987 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 69.931861651999995, 2024-04-26T04:33:28.987 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20488266699999999, 2024-04-26T04:33:28.987 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:28.987 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:28.987 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:28.988 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:28.988 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.988 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:28.988 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:33:28.988 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.988 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.988 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:28.988 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945174+0000", 2024-04-26T04:33:28.988 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:28.988 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.988 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.989 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:28.989 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945176+0000", 2024-04-26T04:33:28.989 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8470000000000001e-06 2024-04-26T04:33:28.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.989 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.989 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:28.989 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945193+0000", 2024-04-26T04:33:28.989 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7013000000000001e-05 2024-04-26T04:33:28.989 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.989 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.990 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:28.990 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945221+0000", 2024-04-26T04:33:28.990 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7670000000000001e-05 2024-04-26T04:33:28.990 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.990 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.990 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:28.990 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:18.945238+0000", 2024-04-26T04:33:28.990 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7113e-05 2024-04-26T04:33:28.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.991 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.991 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:28.991 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.147614+0000", 2024-04-26T04:33:28.991 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202375893 2024-04-26T04:33:28.991 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.991 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.991 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:28.991 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.147669+0000", 2024-04-26T04:33:28.992 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.5729000000000001e-05 2024-04-26T04:33:28.992 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.992 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.992 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:28.992 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150011+0000", 2024-04-26T04:33:28.992 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.002342083 2024-04-26T04:33:28.992 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:28.993 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:28.993 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:28.993 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:19.150057+0000", 2024-04-26T04:33:28.993 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.5318999999999998e-05 2024-04-26T04:33:28.993 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.993 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.994 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.994 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:28.994 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:28.994 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:28.996 INFO:teuthology.orchestra.run.smithi012.stdout:osd.9: {} 2024-04-26T04:33:28.996 INFO:teuthology.orchestra.run.smithi012.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-26T04:33:28.997 INFO:teuthology.orchestra.run.smithi012.stdout:osd.10: {} 2024-04-26T04:33:28.998 INFO:teuthology.orchestra.run.smithi012.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-26T04:33:28.998 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-26T04:33:28.999 INFO:teuthology.orchestra.run.smithi012.stdout:osd.11: {} 2024-04-26T04:33:28.999 INFO:teuthology.orchestra.run.smithi012.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-26T04:33:29.000 INFO:teuthology.orchestra.run.smithi012.stdout:osd.12: {} 2024-04-26T04:33:29.000 INFO:teuthology.orchestra.run.smithi012.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-26T04:33:29.000 INFO:teuthology.orchestra.run.smithi012.stdout:osd.13: {} 2024-04-26T04:33:29.001 INFO:teuthology.orchestra.run.smithi012.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-26T04:33:29.001 INFO:tasks.ceph.osd.4.smithi012.stderr:2024-04-26T04:33:28.989+0000 7fc5ac72a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25080) UID: 0 2024-04-26T04:33:29.010 INFO:teuthology.orchestra.run.smithi012.stdout:osd.14: {} 2024-04-26T04:33:29.011 INFO:teuthology.orchestra.run.smithi012.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-26T04:33:29.045 INFO:teuthology.orchestra.run.smithi012.stdout:osd.15: {} 2024-04-26T04:33:29.045 INFO:teuthology.orchestra.run.smithi012.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-26T04:33:29.096 INFO:tasks.ceph.osd.14.smithi137.stderr:2024-04-26T04:33:29.095+0000 7f376d8e5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25093) UID: 0 2024-04-26T04:33:29.127 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:29.128 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:29.128 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:33:29.128 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:29.142 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-26T04:33:29.197 INFO:tasks.ceph.osd.1.smithi113.stderr:2024-04-26T04:33:29.192+0000 7f667ee90640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25070) UID: 0 2024-04-26T04:33:29.298 INFO:tasks.ceph.osd.11.smithi178.stderr:2024-04-26T04:33:29.293+0000 7f760603d640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25354) UID: 0 2024-04-26T04:33:29.309 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:29.309 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:29.309 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:33:29.309 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:33:29.309 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:29.323 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-26T04:33:29.397 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-04-26T04:33:29.398 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'unique_pool_0' removed 2024-04-26T04:33:29.399 INFO:tasks.ceph.osd.2.smithi137.stderr:2024-04-26T04:33:29.399+0000 7f42493b8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25087) UID: 0 2024-04-26T04:33:29.423 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-04-26T04:33:29.432 INFO:tasks.thrashosds:joining thrashosds 2024-04-26T04:33:29.447 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:29.447 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:33:29.447 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:33:29.447 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:33:29.447 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.447 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 32/30 ECSubWrite(tid=0, reqid=client.4670.0:1, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:29.447 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.202677+0000", 2024-04-26T04:33:29.447 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 64.243612929999998, 2024-04-26T04:33:29.447 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19503779500000001, 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202677+0000", 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202677+0000", 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.448 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202678+0000", 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8599999999999998e-07 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202736+0000", 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.7933000000000003e-05 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202761+0000", 2024-04-26T04:33:29.449 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4706999999999999e-05 2024-04-26T04:33:29.450 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.450 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.450 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.450 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.202786+0000", 2024-04-26T04:33:29.450 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5528999999999999e-05 2024-04-26T04:33:29.450 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.450 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.450 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.450 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.304898+0000", 2024-04-26T04:33:29.450 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102111946 2024-04-26T04:33:29.450 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.450 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.450 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.304915+0000", 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6846000000000001e-05 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397689+0000", 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.092774051999999996 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.397715+0000", 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6196000000000001e-05 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.451 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.452 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.452 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.452 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.452 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 36/30 ECSubWrite(tid=39, reqid=client.4670.0:247, at_version=36'25, trim_to=0'0, roll_forward_to=35'24))", 2024-04-26T04:33:29.452 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.074094+0000", 2024-04-26T04:33:29.452 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 59.372196166999998, 2024-04-26T04:33:29.452 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20298102000000001, 2024-04-26T04:33:29.452 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.452 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.452 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.452 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.452 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.452 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.453 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074094+0000", 2024-04-26T04:33:29.453 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.453 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.453 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.453 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074094+0000", 2024-04-26T04:33:29.453 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.453 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.453 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.453 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074096+0000", 2024-04-26T04:33:29.453 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.854e-06 2024-04-26T04:33:29.453 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.454 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.454 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.454 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074602+0000", 2024-04-26T04:33:29.454 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00050626899999999995 2024-04-26T04:33:29.454 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.454 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.454 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.454 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.074642+0000", 2024-04-26T04:33:29.455 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0113e-05 2024-04-26T04:33:29.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.455 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.455 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.455 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174816+0000", 2024-04-26T04:33:29.455 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10017388200000001 2024-04-26T04:33:29.455 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.455 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.455 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.455 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174981+0000", 2024-04-26T04:33:29.455 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016466500000000001 2024-04-26T04:33:29.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.456 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.456 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.456 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.174995+0000", 2024-04-26T04:33:29.456 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.419e-05 2024-04-26T04:33:29.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.456 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.456 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.456 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.277024+0000", 2024-04-26T04:33:29.456 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102028498 2024-04-26T04:33:29.456 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.456 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.456 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.277075+0000", 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1548999999999997e-05 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 40/30 ECSubWrite(tid=104, reqid=client.4670.0:321, at_version=40'18, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.611130+0000", 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 54.835160278000004, 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201625046, 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.457 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.458 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.458 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.458 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.458 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611130+0000", 2024-04-26T04:33:29.458 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.458 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.458 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.458 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611130+0000", 2024-04-26T04:33:29.458 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.458 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.458 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611131+0000", 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1300000000000001e-07 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611749+0000", 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00061894499999999998 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611779+0000", 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9969999999999999e-05 2024-04-26T04:33:29.459 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.460 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.460 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.460 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.611795+0000", 2024-04-26T04:33:29.460 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5443000000000001e-05 2024-04-26T04:33:29.460 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.460 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.460 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.460 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613563+0000", 2024-04-26T04:33:29.460 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001768211 2024-04-26T04:33:29.460 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.460 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.460 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.460 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613590+0000", 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6633000000000001e-05 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812660+0000", 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19907063799999999 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812755+0000", 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4792999999999999e-05 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.461 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.462 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.462 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.462 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 40/30 ECSubWrite(tid=106, reqid=client.4670.0:322, at_version=40'19, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:33:29.462 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.613168+0000", 2024-04-26T04:33:29.462 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 54.833121962999996, 2024-04-26T04:33:29.462 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.199657323, 2024-04-26T04:33:29.462 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.462 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.462 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.462 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.462 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.462 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.463 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613168+0000", 2024-04-26T04:33:29.463 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.463 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.463 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.463 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613168+0000", 2024-04-26T04:33:29.463 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.463 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.463 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.464 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.464 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613174+0000", 2024-04-26T04:33:29.464 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.9569999999999999e-06 2024-04-26T04:33:29.464 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.464 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.464 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.464 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613348+0000", 2024-04-26T04:33:29.464 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000173535 2024-04-26T04:33:29.465 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.465 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.465 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.465 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613377+0000", 2024-04-26T04:33:29.465 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9522999999999998e-05 2024-04-26T04:33:29.465 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.465 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.465 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.466 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613394+0000", 2024-04-26T04:33:29.466 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.628e-05 2024-04-26T04:33:29.466 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.466 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.466 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.466 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.615501+0000", 2024-04-26T04:33:29.466 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021068710000000002 2024-04-26T04:33:29.466 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.466 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.467 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.467 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.615545+0000", 2024-04-26T04:33:29.467 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4406999999999997e-05 2024-04-26T04:33:29.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.467 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.467 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.467 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812776+0000", 2024-04-26T04:33:29.467 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19723128400000001 2024-04-26T04:33:29.467 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.467 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.467 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812826+0000", 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9465999999999997e-05 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 40/30 ECSubWrite(tid=107, reqid=client.4670.0:323, at_version=40'20, trim_to=0'0, roll_forward_to=36'16))", 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:34.613975+0000", 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 54.832315743000002, 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.199514147, 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.468 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613975+0000", 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613975+0000", 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.469 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.613976+0000", 2024-04-26T04:33:29.470 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8029999999999999e-06 2024-04-26T04:33:29.470 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.470 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.470 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.470 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.614012+0000", 2024-04-26T04:33:29.470 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5283000000000001e-05 2024-04-26T04:33:29.470 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.470 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.470 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.614038+0000", 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5890000000000001e-05 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.614070+0000", 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1992999999999997e-05 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812143+0000", 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.1980729 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.471 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.472 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.472 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.812157+0000", 2024-04-26T04:33:29.472 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.472e-05 2024-04-26T04:33:29.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.472 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.472 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.472 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.813409+0000", 2024-04-26T04:33:29.472 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012518290000000001 2024-04-26T04:33:29.472 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.472 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.473 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.473 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:34.813489+0000", 2024-04-26T04:33:29.473 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9728999999999995e-05 2024-04-26T04:33:29.473 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.473 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.473 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.473 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.473 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.473 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.5s6 56/30 ECSubWrite(tid=63, reqid=client.4670.0:480, at_version=0'0, trim_to=0'0, roll_forward_to=56'44))", 2024-04-26T04:33:29.473 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:50.926782+0000", 2024-04-26T04:33:29.473 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 38.519508479000002, 2024-04-26T04:33:29.473 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20180579100000001, 2024-04-26T04:33:29.473 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.474 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.474 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.474 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.474 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.474 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.474 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926782+0000", 2024-04-26T04:33:29.474 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.474 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.474 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.474 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.474 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926782+0000", 2024-04-26T04:33:29.474 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.475 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.475 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.475 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.475 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926784+0000", 2024-04-26T04:33:29.475 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8530000000000001e-06 2024-04-26T04:33:29.475 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.475 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.475 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.475 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926806+0000", 2024-04-26T04:33:29.476 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2104e-05 2024-04-26T04:33:29.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.476 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.476 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.476 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:50.926835+0000", 2024-04-26T04:33:29.476 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9635999999999999e-05 2024-04-26T04:33:29.476 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.476 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.476 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.476 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.026977+0000", 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014184800000001 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.127264+0000", 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100287105 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.127291+0000", 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7046000000000002e-05 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.477 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.478 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.128508+0000", 2024-04-26T04:33:29.478 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012162760000000001 2024-04-26T04:33:29.478 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.478 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.478 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.478 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.128588+0000", 2024-04-26T04:33:29.478 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9923e-05 2024-04-26T04:33:29.478 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.478 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.478 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.478 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.478 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.478 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 60/30 ECSubWrite(tid=222, reqid=client.4670.0:597, at_version=60'46, trim_to=0'0, roll_forward_to=58'43))", 2024-04-26T04:33:29.479 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:55.982118+0000", 2024-04-26T04:33:29.479 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 33.464172105000003, 2024-04-26T04:33:29.479 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30298714199999999, 2024-04-26T04:33:29.479 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.479 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.479 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.479 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.479 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.479 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.479 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982118+0000", 2024-04-26T04:33:29.479 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.479 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.479 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.480 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.480 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982118+0000", 2024-04-26T04:33:29.480 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.480 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.480 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.480 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.480 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982119+0000", 2024-04-26T04:33:29.480 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1969999999999999e-06 2024-04-26T04:33:29.480 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.480 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.480 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.480 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982416+0000", 2024-04-26T04:33:29.480 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000296087 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982433+0000", 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6986999999999998e-05 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:55.982444+0000", 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1596e-05 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.481 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.182651+0000", 2024-04-26T04:33:29.482 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200207255 2024-04-26T04:33:29.482 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.482 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.482 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.482 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.182661+0000", 2024-04-26T04:33:29.482 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0017000000000001e-05 2024-04-26T04:33:29.482 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.482 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.482 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.482 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.285076+0000", 2024-04-26T04:33:29.482 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102415038 2024-04-26T04:33:29.482 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.482 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.483 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.483 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.285105+0000", 2024-04-26T04:33:29.483 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8965000000000001e-05 2024-04-26T04:33:29.483 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.483 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.483 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.483 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.483 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.483 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 60/30 ECSubWrite(tid=223, reqid=client.4670.0:598, at_version=60'47, trim_to=0'0, roll_forward_to=58'43))", 2024-04-26T04:33:29.483 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:56.083100+0000", 2024-04-26T04:33:29.483 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 33.363190564999996, 2024-04-26T04:33:29.483 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202036665, 2024-04-26T04:33:29.484 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.484 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.484 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.484 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.484 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.484 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.484 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083100+0000", 2024-04-26T04:33:29.484 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.484 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.485 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.485 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.485 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083100+0000", 2024-04-26T04:33:29.485 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.485 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.485 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.485 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.485 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083102+0000", 2024-04-26T04:33:29.485 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2170000000000001e-06 2024-04-26T04:33:29.485 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.485 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.485 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.485 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083219+0000", 2024-04-26T04:33:29.486 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000117049 2024-04-26T04:33:29.486 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.486 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.486 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.486 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.083244+0000", 2024-04-26T04:33:29.486 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4471e-05 2024-04-26T04:33:29.487 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.487 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.487 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.487 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.183323+0000", 2024-04-26T04:33:29.487 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100079945 2024-04-26T04:33:29.487 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.487 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.488 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.488 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.183491+0000", 2024-04-26T04:33:29.488 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016799800000000001 2024-04-26T04:33:29.488 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.488 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.488 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.488 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.183505+0000", 2024-04-26T04:33:29.489 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3740999999999999e-05 2024-04-26T04:33:29.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.489 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.489 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.489 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.285116+0000", 2024-04-26T04:33:29.489 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10161113500000001 2024-04-26T04:33:29.490 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.490 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.490 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.490 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.285136+0000", 2024-04-26T04:33:29.490 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0109e-05 2024-04-26T04:33:29.490 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.491 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.491 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.491 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.491 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 61/30 ECSubWrite(tid=236, reqid=client.4670.0:599, at_version=0'0, trim_to=0'0, roll_forward_to=60'48))", 2024-04-26T04:33:29.491 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:56.591417+0000", 2024-04-26T04:33:29.491 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.854873652000002, 2024-04-26T04:33:29.492 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200740057, 2024-04-26T04:33:29.492 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.492 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.492 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.492 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.493 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.493 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.493 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.591417+0000", 2024-04-26T04:33:29.493 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.493 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.493 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.494 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.494 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.591417+0000", 2024-04-26T04:33:29.494 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.494 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.494 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.494 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.494 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.591418+0000", 2024-04-26T04:33:29.495 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4890000000000001e-06 2024-04-26T04:33:29.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.495 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.495 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.495 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.591439+0000", 2024-04-26T04:33:29.495 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0653999999999999e-05 2024-04-26T04:33:29.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.496 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.496 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.496 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.591463+0000", 2024-04-26T04:33:29.496 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4223000000000002e-05 2024-04-26T04:33:29.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.497 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.497 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.497 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.691580+0000", 2024-04-26T04:33:29.497 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100116592 2024-04-26T04:33:29.497 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.497 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.497 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.498 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.691750+0000", 2024-04-26T04:33:29.498 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017070099999999999 2024-04-26T04:33:29.498 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.498 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.498 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.498 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.691760+0000", 2024-04-26T04:33:29.498 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8670000000000006e-06 2024-04-26T04:33:29.498 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.498 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.498 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.499 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.792110+0000", 2024-04-26T04:33:29.499 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100349255 2024-04-26T04:33:29.499 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.499 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.499 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.499 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.792157+0000", 2024-04-26T04:33:29.499 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7275999999999999e-05 2024-04-26T04:33:29.499 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.499 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.499 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.499 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.499 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.500 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.5s6 67/30 ECSubWrite(tid=86, reqid=client.4670.0:732, at_version=67'59, trim_to=0'0, roll_forward_to=66'54))", 2024-04-26T04:33:29.500 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:02.611645+0000", 2024-04-26T04:33:29.500 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.834645277, 2024-04-26T04:33:29.500 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20163228699999999, 2024-04-26T04:33:29.500 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.500 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.500 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.500 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.500 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.500 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.500 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.611645+0000", 2024-04-26T04:33:29.500 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.500 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.501 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.501 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.501 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.611645+0000", 2024-04-26T04:33:29.501 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.501 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.501 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.501 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.501 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.611647+0000", 2024-04-26T04:33:29.501 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7069999999999999e-06 2024-04-26T04:33:29.501 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.501 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.502 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.502 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.611746+0000", 2024-04-26T04:33:29.502 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.9269000000000001e-05 2024-04-26T04:33:29.502 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.502 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.502 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.502 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.611773+0000", 2024-04-26T04:33:29.502 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7076e-05 2024-04-26T04:33:29.502 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.502 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.502 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.502 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.711865+0000", 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10009203899999999 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.812102+0000", 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100236353 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.812119+0000", 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7552999999999999e-05 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.503 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.813198+0000", 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001078437 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.813277+0000", 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9852999999999996e-05 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.es5 68/30 ECSubWrite(tid=242, reqid=osd.13.0:242, at_version=68'48, trim_to=0'0, roll_forward_to=66'46))", 2024-04-26T04:33:29.504 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.242472+0000", 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.203818015, 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201355757, 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.242472+0000", 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.505 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.242472+0000", 2024-04-26T04:33:29.506 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.506 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.506 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.506 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.506 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.242473+0000", 2024-04-26T04:33:29.506 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0699999999999997e-07 2024-04-26T04:33:29.506 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.506 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.506 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.506 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.242526+0000", 2024-04-26T04:33:29.506 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2706e-05 2024-04-26T04:33:29.506 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.506 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.507 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.507 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.242550+0000", 2024-04-26T04:33:29.507 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4036000000000002e-05 2024-04-26T04:33:29.507 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.507 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.507 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.507 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.242570+0000", 2024-04-26T04:33:29.507 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0653999999999999e-05 2024-04-26T04:33:29.507 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.507 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.508 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.508 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243563+0000", 2024-04-26T04:33:29.508 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00099253699999999993 2024-04-26T04:33:29.508 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.508 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.508 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.508 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243577+0000", 2024-04-26T04:33:29.508 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.447e-05 2024-04-26T04:33:29.508 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.508 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.508 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.443743+0000", 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20016622100000001 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.443828+0000", 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.4625999999999999e-05 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.es5 68/30 ECSubWrite(tid=243, reqid=osd.13.0:243, at_version=68'50, trim_to=0'0, roll_forward_to=66'46))", 2024-04-26T04:33:29.509 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.243314+0000", 2024-04-26T04:33:29.510 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.202976326000002, 2024-04-26T04:33:29.510 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20115462200000001, 2024-04-26T04:33:29.510 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.510 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.510 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.510 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.510 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.511 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.511 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243314+0000", 2024-04-26T04:33:29.511 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.511 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.511 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.511 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.511 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243314+0000", 2024-04-26T04:33:29.511 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.511 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.511 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.511 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.511 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243316+0000", 2024-04-26T04:33:29.512 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.773e-06 2024-04-26T04:33:29.512 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.512 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.512 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.512 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243355+0000", 2024-04-26T04:33:29.512 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8733999999999999e-05 2024-04-26T04:33:29.512 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.512 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.512 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.512 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243385+0000", 2024-04-26T04:33:29.512 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0372999999999999e-05 2024-04-26T04:33:29.512 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.512 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.513 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.513 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.243404+0000", 2024-04-26T04:33:29.513 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8943000000000001e-05 2024-04-26T04:33:29.513 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.513 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.513 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.513 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.442893+0000", 2024-04-26T04:33:29.513 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.199489097 2024-04-26T04:33:29.513 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.513 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.442924+0000", 2024-04-26T04:33:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1142000000000001e-05 2024-04-26T04:33:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.444396+0000", 2024-04-26T04:33:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001472101 2024-04-26T04:33:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.444469+0000", 2024-04-26T04:33:29.514 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2459000000000005e-05 2024-04-26T04:33:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.es5 68/30 ECSubWrite(tid=244, reqid=osd.13.0:243, at_version=0'0, trim_to=0'0, roll_forward_to=68'50))", 2024-04-26T04:33:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.645143+0000", 2024-04-26T04:33:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 25.801147493999999, 2024-04-26T04:33:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20052077200000001, 2024-04-26T04:33:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.515 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.645143+0000", 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.645143+0000", 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.645145+0000", 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6869999999999999e-06 2024-04-26T04:33:29.516 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.517 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.517 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.517 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.645166+0000", 2024-04-26T04:33:29.518 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1046000000000001e-05 2024-04-26T04:33:29.518 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.518 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.518 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.519 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.645195+0000", 2024-04-26T04:33:29.519 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9104000000000001e-05 2024-04-26T04:33:29.519 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.519 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.519 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.519 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.645210+0000", 2024-04-26T04:33:29.519 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5662999999999999e-05 2024-04-26T04:33:29.520 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.520 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.520 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.520 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.844505+0000", 2024-04-26T04:33:29.520 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19929448499999999 2024-04-26T04:33:29.520 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.520 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.520 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.520 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.844520+0000", 2024-04-26T04:33:29.520 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4786999999999999e-05 2024-04-26T04:33:29.520 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.520 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.520 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.845589+0000", 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001069637 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.845664+0000", 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4363000000000006e-05 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.5s6 68/30 ECSubWrite(tid=91, reqid=osd.15.0:90, at_version=0'0, trim_to=0'0, roll_forward_to=68'63))", 2024-04-26T04:33:29.521 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.835304+0000", 2024-04-26T04:33:29.522 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 25.610986531999998, 2024-04-26T04:33:29.522 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20152138999999999, 2024-04-26T04:33:29.522 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.522 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.522 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.522 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.522 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.522 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.522 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.835304+0000", 2024-04-26T04:33:29.522 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.522 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.522 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.522 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.523 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.835304+0000", 2024-04-26T04:33:29.523 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.523 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.523 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.523 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.523 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.835306+0000", 2024-04-26T04:33:29.523 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7999999999999999e-06 2024-04-26T04:33:29.523 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.523 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.523 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.524 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.835327+0000", 2024-04-26T04:33:29.524 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1250000000000002e-05 2024-04-26T04:33:29.524 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.524 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.524 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.524 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.835358+0000", 2024-04-26T04:33:29.524 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.065e-05 2024-04-26T04:33:29.524 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.524 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.524 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.524 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.935467+0000", 2024-04-26T04:33:29.525 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100109771 2024-04-26T04:33:29.525 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.525 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.525 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.525 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.035772+0000", 2024-04-26T04:33:29.525 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100304755 2024-04-26T04:33:29.525 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.525 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.525 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.525 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.035787+0000", 2024-04-26T04:33:29.525 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4953000000000001e-05 2024-04-26T04:33:29.525 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.525 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.526 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.526 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.036756+0000", 2024-04-26T04:33:29.526 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00096854099999999995 2024-04-26T04:33:29.526 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.526 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.526 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.526 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.036825+0000", 2024-04-26T04:33:29.526 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9670000000000002e-05 2024-04-26T04:33:29.526 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.526 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.526 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.526 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.526 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.527 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.es5 79/30 ECSubWrite(tid=274, reqid=osd.13.0:274, at_version=79'78, trim_to=0'0, roll_forward_to=78'74))", 2024-04-26T04:33:29.527 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:14.467190+0000", 2024-04-26T04:33:29.527 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 14.979100138, 2024-04-26T04:33:29.527 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.202128577, 2024-04-26T04:33:29.527 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.527 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.527 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.527 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.527 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.527 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.527 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.467190+0000", 2024-04-26T04:33:29.527 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.527 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.528 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.528 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.528 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.467190+0000", 2024-04-26T04:33:29.528 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.528 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.528 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.528 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.528 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.467192+0000", 2024-04-26T04:33:29.528 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.827e-06 2024-04-26T04:33:29.529 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.529 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.529 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.529 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.467267+0000", 2024-04-26T04:33:29.529 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5216000000000006e-05 2024-04-26T04:33:29.529 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.529 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.529 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.529 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.467297+0000", 2024-04-26T04:33:29.529 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0116000000000001e-05 2024-04-26T04:33:29.529 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.530 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.530 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.530 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.567433+0000", 2024-04-26T04:33:29.530 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100135295 2024-04-26T04:33:29.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.530 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.530 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.530 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.667670+0000", 2024-04-26T04:33:29.530 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10023686699999999 2024-04-26T04:33:29.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.531 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.531 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.531 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.667713+0000", 2024-04-26T04:33:29.531 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.3093999999999998e-05 2024-04-26T04:33:29.531 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.531 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.532 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.532 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.669212+0000", 2024-04-26T04:33:29.532 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00149971 2024-04-26T04:33:29.532 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.532 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.532 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.533 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.669319+0000", 2024-04-26T04:33:29.533 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000106452 2024-04-26T04:33:29.533 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.533 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.533 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.533 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.534 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.534 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.7s1 83/30 ECSubWrite(tid=380, reqid=osd.4.0:377, at_version=0'0, trim_to=0'0, roll_forward_to=83'82))", 2024-04-26T04:33:29.534 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:18.749182+0000", 2024-04-26T04:33:29.534 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 10.697108862, 2024-04-26T04:33:29.534 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.301230463, 2024-04-26T04:33:29.534 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.535 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.535 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.535 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.535 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.535 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.535 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.749182+0000", 2024-04-26T04:33:29.535 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.535 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.535 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.535 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.535 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.749182+0000", 2024-04-26T04:33:29.536 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.536 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.536 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.536 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.536 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.749191+0000", 2024-04-26T04:33:29.536 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.3859999999999996e-06 2024-04-26T04:33:29.536 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.536 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.536 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.536 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.749212+0000", 2024-04-26T04:33:29.536 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1083000000000001e-05 2024-04-26T04:33:29.537 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.537 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.537 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.537 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.749237+0000", 2024-04-26T04:33:29.537 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5423999999999999e-05 2024-04-26T04:33:29.537 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.537 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.537 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.537 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.849334+0000", 2024-04-26T04:33:29.537 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100096544 2024-04-26T04:33:29.537 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.537 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.537 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.538 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.949594+0000", 2024-04-26T04:33:29.538 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100259731 2024-04-26T04:33:29.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.538 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.538 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.538 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.949612+0000", 2024-04-26T04:33:29.538 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7927e-05 2024-04-26T04:33:29.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.538 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.539 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.539 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.050316+0000", 2024-04-26T04:33:29.539 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100703959 2024-04-26T04:33:29.539 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.539 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.539 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.539 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.050412+0000", 2024-04-26T04:33:29.539 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6409000000000005e-05 2024-04-26T04:33:29.539 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.539 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.539 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.539 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.540 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.540 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.es5 84/30 ECSubWrite(tid=292, reqid=client.4670.0:944, at_version=84'101, trim_to=0'0, roll_forward_to=83'94))", 2024-04-26T04:33:29.540 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:19.569584+0000", 2024-04-26T04:33:29.540 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 9.8767062459999995, 2024-04-26T04:33:29.540 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20408479900000001, 2024-04-26T04:33:29.540 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.540 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.540 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.540 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.569584+0000", 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.569584+0000", 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.569585+0000", 2024-04-26T04:33:29.541 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9999999999999997e-07 2024-04-26T04:33:29.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.542 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.542 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.542 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.569781+0000", 2024-04-26T04:33:29.542 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00019659500000000001 2024-04-26T04:33:29.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.542 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.542 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.542 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.569797+0000", 2024-04-26T04:33:29.542 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5979999999999999e-05 2024-04-26T04:33:29.542 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.542 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.542 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.543 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.669940+0000", 2024-04-26T04:33:29.543 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100142893 2024-04-26T04:33:29.543 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.543 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.543 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.543 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.770228+0000", 2024-04-26T04:33:29.543 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10028809 2024-04-26T04:33:29.543 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.543 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.543 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.543 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.770255+0000", 2024-04-26T04:33:29.543 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6270000000000001e-05 2024-04-26T04:33:29.543 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.773576+0000", 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0033215150000000001 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.773669+0000", 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2756000000000004e-05 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.544 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.545 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.bs4 89/30 ECSubWrite(tid=407, reqid=osd.4.0:407, at_version=89'100, trim_to=0'0, roll_forward_to=88'98))", 2024-04-26T04:33:29.545 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:24.456853+0000", 2024-04-26T04:33:29.545 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.9894373080000003, 2024-04-26T04:33:29.545 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.19818708900000001, 2024-04-26T04:33:29.545 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.545 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.545 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.545 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.545 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.545 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.545 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.456853+0000", 2024-04-26T04:33:29.545 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.545 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.546 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.546 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.546 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.456853+0000", 2024-04-26T04:33:29.546 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.546 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.546 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.546 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.546 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.456853+0000", 2024-04-26T04:33:29.546 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9999999999999998e-07 2024-04-26T04:33:29.546 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.546 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.456888+0000", 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4122999999999998e-05 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.456910+0000", 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2883000000000001e-05 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.456927+0000", 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6249999999999999e-05 2024-04-26T04:33:29.547 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.557232+0000", 2024-04-26T04:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100305353 2024-04-26T04:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.557255+0000", 2024-04-26T04:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3410000000000001e-05 2024-04-26T04:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.548 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.654987+0000", 2024-04-26T04:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.097731410000000005 2024-04-26T04:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.655040+0000", 2024-04-26T04:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3260000000000002e-05 2024-04-26T04:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.549 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 89/30 ECSubWrite(tid=165, reqid=osd.5.0:164, at_version=0'0, trim_to=0'0, roll_forward_to=89'144))", 2024-04-26T04:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:24.756704+0000", 2024-04-26T04:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.6895864840000003, 2024-04-26T04:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30261255199999998, 2024-04-26T04:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.550 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.756704+0000", 2024-04-26T04:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.551 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.756704+0000", 2024-04-26T04:33:29.553 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.756706+0000", 2024-04-26T04:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5840000000000002e-06 2024-04-26T04:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.756734+0000", 2024-04-26T04:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8005999999999999e-05 2024-04-26T04:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.554 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.756776+0000", 2024-04-26T04:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1950000000000003e-05 2024-04-26T04:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.856927+0000", 2024-04-26T04:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100150105 2024-04-26T04:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.555 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.957146+0000", 2024-04-26T04:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100219428 2024-04-26T04:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.957162+0000", 2024-04-26T04:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5744e-05 2024-04-26T04:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:25.059242+0000", 2024-04-26T04:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102080135 2024-04-26T04:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.556 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:25.059316+0000", 2024-04-26T04:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4599999999999997e-05 2024-04-26T04:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s6 89/30 ECSubWrite(tid=166, reqid=osd.5.0:166, at_version=89'146, trim_to=0'0, roll_forward_to=89'144))", 2024-04-26T04:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:24.856992+0000", 2024-04-26T04:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 4.5892982099999999, 2024-04-26T04:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.203428305, 2024-04-26T04:33:29.557 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.856992+0000", 2024-04-26T04:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.856992+0000", 2024-04-26T04:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:29.558 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.856994+0000", 2024-04-26T04:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3400000000000001e-06 2024-04-26T04:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.857046+0000", 2024-04-26T04:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2349999999999999e-05 2024-04-26T04:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:29.559 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.857081+0000", 2024-04-26T04:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5123000000000002e-05 2024-04-26T04:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.857104+0000", 2024-04-26T04:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3346e-05 2024-04-26T04:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:29.560 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:25.058416+0000", 2024-04-26T04:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201311716 2024-04-26T04:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:25.058431+0000", 2024-04-26T04:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4800000000000001e-05 2024-04-26T04:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:29.561 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:25.060327+0000", 2024-04-26T04:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0018962810000000001 2024-04-26T04:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:25.060420+0000", 2024-04-26T04:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.3349000000000004e-05 2024-04-26T04:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:29.562 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:29.563 INFO:tasks.thrashosds.thrasher:inning osd 2024-04-26T04:33:29.563 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-04-26T04:33:29.563 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-04-26T04:33:29.565 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-26T04:33:29.735 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:29.735 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:29.735 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:33:29.735 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:29.749 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-26T04:33:29.865 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:29.866 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:29.866 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:33:29.866 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:33:29.866 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:29.880 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-26T04:33:29.881 INFO:tasks.ceph.mon.a.smithi012.stderr:2024-04-26T04:33:29.873+0000 7f05ec032640 -1 mon.a@0(leader).osd e94 definitely_dead 0 2024-04-26T04:33:30.000 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:30.000 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:33:30.001 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:33:30.001 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:33:30.001 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.001 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 35/30 ECSubWrite(tid=61, reqid=client.4670.0:211, at_version=0'0, trim_to=0'0, roll_forward_to=35'17))", 2024-04-26T04:33:30.001 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:29.330344+0000", 2024-04-26T04:33:30.001 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 60.669306487999997, 2024-04-26T04:33:30.001 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105013624, 2024-04-26T04:33:30.001 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.001 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.001 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.001 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.001 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.001 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.002 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330344+0000", 2024-04-26T04:33:30.002 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.002 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.002 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.002 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.002 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330344+0000", 2024-04-26T04:33:30.002 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.002 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.002 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.002 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.002 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330346+0000", 2024-04-26T04:33:30.002 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1399999999999998e-06 2024-04-26T04:33:30.002 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330371+0000", 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5320000000000002e-05 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330403+0000", 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2305999999999999e-05 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.330423+0000", 2024-04-26T04:33:30.003 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9179000000000001e-05 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.433931+0000", 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103508818 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.433962+0000", 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0576e-05 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.435260+0000", 2024-04-26T04:33:30.004 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001297819 2024-04-26T04:33:30.005 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.005 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.005 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.005 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:29.435357+0000", 2024-04-26T04:33:30.005 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.7466000000000005e-05 2024-04-26T04:33:30.005 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.005 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.005 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.005 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.005 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.005 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 36/30 ECSubWrite(tid=74, reqid=client.4670.0:260, at_version=36'28, trim_to=0'0, roll_forward_to=36'19))", 2024-04-26T04:33:30.005 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.258165+0000", 2024-04-26T04:33:30.005 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 59.741485376999997, 2024-04-26T04:33:30.006 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10329530100000001, 2024-04-26T04:33:30.006 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.006 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.006 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.006 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.006 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.006 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.006 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258165+0000", 2024-04-26T04:33:30.006 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.006 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.007 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.007 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.007 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258165+0000", 2024-04-26T04:33:30.007 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.007 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.007 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.007 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.007 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258166+0000", 2024-04-26T04:33:30.007 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8299999999999995e-07 2024-04-26T04:33:30.007 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.007 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.007 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.007 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258269+0000", 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000103069 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258288+0000", 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8652999999999999e-05 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.258299+0000", 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1243999999999999e-05 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.008 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.359776+0000", 2024-04-26T04:33:30.009 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10147722000000001 2024-04-26T04:33:30.009 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.009 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.009 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.009 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.359798+0000", 2024-04-26T04:33:30.009 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2476999999999998e-05 2024-04-26T04:33:30.009 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.009 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.009 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.009 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.361395+0000", 2024-04-26T04:33:30.010 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0015963660000000001 2024-04-26T04:33:30.010 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.010 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.010 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.010 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.361460+0000", 2024-04-26T04:33:30.010 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.5289000000000005e-05 2024-04-26T04:33:30.010 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.010 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.010 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.010 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.010 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.010 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 41/30 ECSubWrite(tid=44, reqid=client.4670.0:346, at_version=41'26, trim_to=0'0, roll_forward_to=38'24))", 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 54.211362319999999, 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103457721, 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.011 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788288+0000", 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4600000000000005e-07 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788918+0000", 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00062948799999999999 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.012 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.788943+0000", 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4706e-05 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.889107+0000", 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100164644 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.889285+0000", 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017809600000000001 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.013 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.889311+0000", 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5233e-05 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.891664+0000", 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0023538460000000001 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.891746+0000", 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.1162000000000007e-05 2024-04-26T04:33:30.014 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.015 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.015 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.015 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.015 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.015 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/30 ECSubWrite(tid=186, reqid=client.4670.0:483, at_version=57'57, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T04:33:30.015 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.517139+0000", 2024-04-26T04:33:30.015 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 38.482511666999997, 2024-04-26T04:33:30.015 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.22991919599999999, 2024-04-26T04:33:30.015 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.015 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.015 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.015 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.016 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.016 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.016 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.517139+0000", 2024-04-26T04:33:30.016 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.016 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.016 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.016 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.016 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.517139+0000", 2024-04-26T04:33:30.016 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.016 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.016 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.016 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.016 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.517139+0000", 2024-04-26T04:33:30.017 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.6700000000000003e-07 2024-04-26T04:33:30.017 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.017 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.017 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.017 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.518064+0000", 2024-04-26T04:33:30.017 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00092505799999999998 2024-04-26T04:33:30.017 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.017 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.017 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.017 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.518095+0000", 2024-04-26T04:33:30.017 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0929999999999997e-05 2024-04-26T04:33:30.017 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.017 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.018 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.018 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.518120+0000", 2024-04-26T04:33:30.018 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4888999999999998e-05 2024-04-26T04:33:30.018 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.018 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.018 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.018 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.644583+0000", 2024-04-26T04:33:30.018 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.12646262999999999 2024-04-26T04:33:30.018 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.018 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.018 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.018 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.644598+0000", 2024-04-26T04:33:30.019 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5427e-05 2024-04-26T04:33:30.019 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.019 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.019 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.019 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747001+0000", 2024-04-26T04:33:30.019 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102403228 2024-04-26T04:33:30.019 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.019 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.019 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.019 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747058+0000", 2024-04-26T04:33:30.019 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6366999999999998e-05 2024-04-26T04:33:30.019 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.019 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/30 ECSubWrite(tid=187, reqid=client.4670.0:484, at_version=57'58, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.519747+0000", 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 38.479903270000001, 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.22735620400000001, 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.020 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.519747+0000", 2024-04-26T04:33:30.021 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.021 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.021 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.021 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.021 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.519747+0000", 2024-04-26T04:33:30.021 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.021 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.021 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.021 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.021 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.519764+0000", 2024-04-26T04:33:30.021 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7249999999999999e-05 2024-04-26T04:33:30.021 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.021 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.520091+0000", 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000326457 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.520114+0000", 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3462999999999999e-05 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.520135+0000", 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0449999999999999e-05 2024-04-26T04:33:30.022 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.646545+0000", 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.12641021699999999 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.646559+0000", 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4137e-05 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747071+0000", 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100512381 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747103+0000", 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1848999999999999e-05 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/30 ECSubWrite(tid=188, reqid=client.4670.0:485, at_version=57'59, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.521190+0000", 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 38.478460169000002, 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.225943687, 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521190+0000", 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521190+0000", 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521191+0000", 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2500000000000001e-06 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521409+0000", 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000218192 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.521431+0000", 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1776e-05 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621545+0000", 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10011374300000001 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.647483+0000", 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.025938086999999999 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.647497+0000", 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.376e-05 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747108+0000", 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.099611379 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747134+0000", 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.55e-05 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 57/30 ECSubWrite(tid=189, reqid=client.4670.0:486, at_version=57'60, trim_to=0'0, roll_forward_to=52'55))", 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:51.621601+0000", 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 38.378049458, 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.126106461, 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621601+0000", 2024-04-26T04:33:30.029 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.030 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.030 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.030 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.030 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621601+0000", 2024-04-26T04:33:30.030 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.030 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.030 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.030 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.030 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621602+0000", 2024-04-26T04:33:30.030 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.576e-06 2024-04-26T04:33:30.030 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621645+0000", 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2620000000000002e-05 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621689+0000", 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4502999999999997e-05 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.621712+0000", 2024-04-26T04:33:30.031 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2486000000000001e-05 2024-04-26T04:33:30.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.032 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.032 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.032 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.746683+0000", 2024-04-26T04:33:30.032 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.124971128 2024-04-26T04:33:30.032 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.032 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.032 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.746698+0000", 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5140000000000001e-05 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747655+0000", 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00095687099999999998 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:51.747707+0000", 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.2136999999999999e-05 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.033 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.034 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.034 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.034 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.034 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 58/30 ECSubWrite(tid=195, reqid=osd.4.0:195, at_version=58'62, trim_to=0'0, roll_forward_to=57'60))", 2024-04-26T04:33:30.034 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:52.382786+0000", 2024-04-26T04:33:30.034 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 37.616864008999997, 2024-04-26T04:33:30.034 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20206534200000001, 2024-04-26T04:33:30.034 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.034 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.034 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.034 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.034 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382786+0000", 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382786+0000", 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382787+0000", 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9000000000000002e-07 2024-04-26T04:33:30.035 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382822+0000", 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.5889999999999997e-05 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.382851+0000", 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8370000000000001e-05 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.482997+0000", 2024-04-26T04:33:30.036 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100146258 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.583261+0000", 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100264055 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.583277+0000", 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5783000000000001e-05 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.037 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.584773+0000", 2024-04-26T04:33:30.038 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014958440000000001 2024-04-26T04:33:30.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.038 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.038 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.038 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:52.584852+0000", 2024-04-26T04:33:30.038 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8751999999999994e-05 2024-04-26T04:33:30.038 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.038 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.038 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.038 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.038 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.038 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpReadReply(3.2s1 61/60 ECSubReadReply(tid=2, attrs_read=0))", 2024-04-26T04:33:30.038 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:56.636057+0000", 2024-04-26T04:33:30.039 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 33.363593606000002, 2024-04-26T04:33:30.039 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200517522, 2024-04-26T04:33:30.039 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.039 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.039 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.039 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.039 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.039 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636057+0000", 2024-04-26T04:33:30.039 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.039 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.039 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636057+0000", 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636058+0000", 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1429999999999999e-06 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636180+0000", 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000122409 2024-04-26T04:33:30.040 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.636199+0000", 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8536999999999999e-05 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.736272+0000", 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100073096 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.836499+0000", 2024-04-26T04:33:30.041 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10022764100000001 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:56.836574+0000", 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4696000000000004e-05 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 67/30 ECSubWrite(tid=66, reqid=client.4670.0:743, at_version=67'44, trim_to=0'0, roll_forward_to=65'40))", 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:02.578744+0000", 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 27.420906635000001, 2024-04-26T04:33:30.042 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10783959799999999, 2024-04-26T04:33:30.043 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.043 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.043 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.043 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.043 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.043 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.043 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.578744+0000", 2024-04-26T04:33:30.043 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.043 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.043 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.043 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.044 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.578744+0000", 2024-04-26T04:33:30.044 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.044 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.044 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.044 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.578745+0000", 2024-04-26T04:33:30.044 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.113e-06 2024-04-26T04:33:30.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.044 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.044 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.044 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.578975+0000", 2024-04-26T04:33:30.044 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00023011099999999999 2024-04-26T04:33:30.044 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.578996+0000", 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.0996999999999999e-05 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.579015+0000", 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8966000000000001e-05 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583750+0000", 2024-04-26T04:33:30.045 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0047352059999999996 2024-04-26T04:33:30.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.046 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.046 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.046 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583767+0000", 2024-04-26T04:33:30.046 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7436999999999999e-05 2024-04-26T04:33:30.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.046 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.046 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.046 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.686514+0000", 2024-04-26T04:33:30.046 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10274667899999999 2024-04-26T04:33:30.046 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.046 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.046 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.686583+0000", 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9089000000000002e-05 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 67/30 ECSubWrite(tid=67, reqid=client.4670.0:744, at_version=67'45, trim_to=0'0, roll_forward_to=65'40))", 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:02.583506+0000", 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 27.416144710000001, 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10420913699999999, 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.047 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.048 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.048 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.048 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.048 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583506+0000", 2024-04-26T04:33:30.048 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.048 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.048 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.048 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583506+0000", 2024-04-26T04:33:30.048 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.048 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.048 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583507+0000", 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8199999999999999e-06 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583576+0000", 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9103000000000006e-05 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583606+0000", 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9626000000000001e-05 2024-04-26T04:33:30.049 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.050 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.050 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.050 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.583622+0000", 2024-04-26T04:33:30.050 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5756999999999998e-05 2024-04-26T04:33:30.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.050 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.050 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.050 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.685678+0000", 2024-04-26T04:33:30.050 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102056015 2024-04-26T04:33:30.050 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.050 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.050 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.050 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.685693+0000", 2024-04-26T04:33:30.051 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4943000000000001e-05 2024-04-26T04:33:30.051 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.051 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.051 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.051 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.687636+0000", 2024-04-26T04:33:30.051 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001943033 2024-04-26T04:33:30.051 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.051 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.051 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.051 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.687715+0000", 2024-04-26T04:33:30.051 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.8839999999999997e-05 2024-04-26T04:33:30.051 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.051 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 67/30 ECSubWrite(tid=68, reqid=client.4670.0:745, at_version=67'46, trim_to=0'0, roll_forward_to=65'40))", 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:02.585301+0000", 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 27.414348822000001, 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10331043099999999, 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.052 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.585301+0000", 2024-04-26T04:33:30.053 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.053 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.053 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.053 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.585301+0000", 2024-04-26T04:33:30.053 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.053 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.053 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.053 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.585303+0000", 2024-04-26T04:33:30.053 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.79e-06 2024-04-26T04:33:30.053 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.585338+0000", 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4459000000000003e-05 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.585363+0000", 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5063999999999999e-05 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.585380+0000", 2024-04-26T04:33:30.054 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7212999999999999e-05 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.686687+0000", 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10130702799999999 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.686714+0000", 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7379999999999999e-05 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.055 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.688535+0000", 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001820781 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.688612+0000", 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6716000000000002e-05 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.1s4 68/30 ECSubWrite(tid=271, reqid=osd.4.0:271, at_version=68'78, trim_to=0'0, roll_forward_to=65'76))", 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.337687+0000", 2024-04-26T04:33:30.056 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.661962767999999, 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103882078, 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.337687+0000", 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.337687+0000", 2024-04-26T04:33:30.057 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.337688+0000", 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.9999999999999998e-07 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.337711+0000", 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3456e-05 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.058 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.337729+0000", 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7631000000000001e-05 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.337739+0000", 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8239999999999995e-06 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.440088+0000", 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102349147 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.059 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.440106+0000", 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.855e-05 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.441496+0000", 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001389728 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.441570+0000", 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.3342000000000004e-05 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.060 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 68/30 ECSubWrite(tid=71, reqid=osd.7.0:71, at_version=68'48, trim_to=0'0, roll_forward_to=67'46))", 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.702942+0000", 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.296708115000001, 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.104726319, 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.702942+0000", 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.061 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.062 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.062 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.062 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.702942+0000", 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.702944+0000", 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.59e-06 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.702991+0000", 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7519999999999999e-05 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.063 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.703025+0000", 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3577000000000001e-05 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.703047+0000", 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2682000000000001e-05 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.803360+0000", 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100312097 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.064 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.803375+0000", 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4949999999999999e-05 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.807578+0000", 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0042032199999999997 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.807668+0000", 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.0682999999999999e-05 2024-04-26T04:33:30.065 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.066 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.066 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.066 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.066 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.066 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 68/30 ECSubWrite(tid=72, reqid=osd.7.0:72, at_version=68'50, trim_to=0'0, roll_forward_to=67'46))", 2024-04-26T04:33:30.066 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.705213+0000", 2024-04-26T04:33:30.066 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.294437414000001, 2024-04-26T04:33:30.066 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10635373400000001, 2024-04-26T04:33:30.066 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.066 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.066 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.066 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.066 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.067 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.067 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705213+0000", 2024-04-26T04:33:30.067 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.067 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.067 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.067 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.067 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705213+0000", 2024-04-26T04:33:30.067 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.067 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.067 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.067 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.068 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705215+0000", 2024-04-26T04:33:30.068 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9630000000000001e-06 2024-04-26T04:33:30.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.068 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.068 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.068 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705265+0000", 2024-04-26T04:33:30.068 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.0073e-05 2024-04-26T04:33:30.068 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.068 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.068 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.068 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705303+0000", 2024-04-26T04:33:30.068 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8206e-05 2024-04-26T04:33:30.069 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.069 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.069 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.069 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705326+0000", 2024-04-26T04:33:30.069 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3453000000000001e-05 2024-04-26T04:33:30.069 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.069 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.069 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.069 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.805555+0000", 2024-04-26T04:33:30.069 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100228715 2024-04-26T04:33:30.069 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.069 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.070 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.070 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.805585+0000", 2024-04-26T04:33:30.070 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9828999999999999e-05 2024-04-26T04:33:30.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.070 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.070 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.070 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.811484+0000", 2024-04-26T04:33:30.070 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0058986029999999997 2024-04-26T04:33:30.070 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.070 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.070 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.071 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.811567+0000", 2024-04-26T04:33:30.071 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2892000000000005e-05 2024-04-26T04:33:30.071 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.071 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.071 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.071 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.071 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.071 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds3 68/30 ECSubWrite(tid=73, reqid=osd.7.0:72, at_version=0'0, trim_to=0'0, roll_forward_to=68'50))", 2024-04-26T04:33:30.071 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:04.010174+0000", 2024-04-26T04:33:30.071 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 25.989476023000002, 2024-04-26T04:33:30.071 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201687914, 2024-04-26T04:33:30.071 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.072 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.072 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.072 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.072 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.072 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.072 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.010174+0000", 2024-04-26T04:33:30.072 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.072 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.072 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.072 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.072 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.010174+0000", 2024-04-26T04:33:30.072 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.073 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.073 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.073 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.010177+0000", 2024-04-26T04:33:30.073 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0429999999999999e-06 2024-04-26T04:33:30.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.073 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.073 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.073 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.010199+0000", 2024-04-26T04:33:30.073 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1494000000000001e-05 2024-04-26T04:33:30.073 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.073 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.010229+0000", 2024-04-26T04:33:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0759999999999997e-05 2024-04-26T04:33:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.110375+0000", 2024-04-26T04:33:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014508599999999 2024-04-26T04:33:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.210624+0000", 2024-04-26T04:33:30.074 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100249067 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.210641+0000", 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7295999999999998e-05 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.211820+0000", 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001179474 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.211862+0000", 2024-04-26T04:33:30.075 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1693999999999998e-05 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 72/30 ECSubWrite(tid=120, reqid=client.4670.0:818, at_version=72'83, trim_to=0'0, roll_forward_to=71'80))", 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:07.490773+0000", 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 22.508877522999999, 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.105442548, 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.076 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.490773+0000", 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.490773+0000", 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.490774+0000", 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7400000000000001e-06 2024-04-26T04:33:30.077 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.491052+0000", 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00027784799999999997 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.491082+0000", 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9573e-05 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.491098+0000", 2024-04-26T04:33:30.078 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6393e-05 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.588489+0000", 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.097390657000000005 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.588504+0000", 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5107e-05 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.079 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.596157+0000", 2024-04-26T04:33:30.080 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0076531239999999999 2024-04-26T04:33:30.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.080 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.080 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.080 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.596215+0000", 2024-04-26T04:33:30.080 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.8106000000000002e-05 2024-04-26T04:33:30.080 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.080 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.080 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.080 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.080 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.080 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 72/30 ECSubWrite(tid=123, reqid=client.4670.0:821, at_version=72'86, trim_to=0'0, roll_forward_to=71'80))", 2024-04-26T04:33:30.080 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:07.689310+0000", 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 22.310340141000001, 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201647774, 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.689310+0000", 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.689310+0000", 2024-04-26T04:33:30.081 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.082 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.082 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.082 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.082 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.689311+0000", 2024-04-26T04:33:30.082 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6000000000000003e-07 2024-04-26T04:33:30.082 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.082 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.082 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.082 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.689353+0000", 2024-04-26T04:33:30.082 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2518999999999997e-05 2024-04-26T04:33:30.082 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.082 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.082 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.689367+0000", 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3720000000000001e-05 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.789502+0000", 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10013504300000001 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.889740+0000", 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10023826199999999 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.083 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.889755+0000", 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4763000000000001e-05 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.890881+0000", 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0011258570000000001 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:07.890958+0000", 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.6849999999999998e-05 2024-04-26T04:33:30.084 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.085 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.085 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.085 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.085 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.085 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 79/30 ECSubWrite(tid=145, reqid=osd.11.0:145, at_version=79'110, trim_to=0'0, roll_forward_to=78'106))", 2024-04-26T04:33:30.085 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:14.457053+0000", 2024-04-26T04:33:30.085 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 15.542597535000001, 2024-04-26T04:33:30.085 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103356644, 2024-04-26T04:33:30.085 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.085 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.085 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.085 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.457053+0000", 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.457053+0000", 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.457055+0000", 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.86e-06 2024-04-26T04:33:30.086 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.457106+0000", 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1560000000000001e-05 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.457136+0000", 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9825999999999999e-05 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.457159+0000", 2024-04-26T04:33:30.087 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3413000000000001e-05 2024-04-26T04:33:30.088 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.088 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.088 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.088 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.457297+0000", 2024-04-26T04:33:30.088 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013787599999999999 2024-04-26T04:33:30.088 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.088 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.088 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.088 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.457313+0000", 2024-04-26T04:33:30.088 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5945999999999999e-05 2024-04-26T04:33:30.088 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.088 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.088 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.560353+0000", 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103039617 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:14.560409+0000", 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.6545999999999997e-05 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.fs5 82/30 ECSubWrite(tid=159, reqid=osd.11.0:159, at_version=82'128, trim_to=0'0, roll_forward_to=82'124))", 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:18.305430+0000", 2024-04-26T04:33:30.089 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 11.694220606, 2024-04-26T04:33:30.090 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.103493084, 2024-04-26T04:33:30.090 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.090 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.090 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.090 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.090 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.090 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.090 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.305430+0000", 2024-04-26T04:33:30.090 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.090 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.090 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.090 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.090 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.305430+0000", 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.305432+0000", 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.99e-06 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.305485+0000", 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3808999999999999e-05 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.091 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.305510+0000", 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4263000000000001e-05 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.305542+0000", 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2672999999999997e-05 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.406687+0000", 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101144888 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.092 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.093 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.093 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.406703+0000", 2024-04-26T04:33:30.093 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6220000000000001e-05 2024-04-26T04:33:30.093 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.093 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.093 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.093 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.408862+0000", 2024-04-26T04:33:30.093 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021587820000000001 2024-04-26T04:33:30.093 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.093 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.093 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.093 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:18.408923+0000", 2024-04-26T04:33:30.093 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.0458999999999998e-05 2024-04-26T04:33:30.094 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.094 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.094 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.094 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.094 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.094 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:30.095 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-26T04:33:30.220 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:30.220 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:30.220 INFO:teuthology.orchestra.run.smithi137.stdout: "num_ops": 0 2024-04-26T04:33:30.221 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:30.234 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-26T04:33:30.356 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:30.356 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [], 2024-04-26T04:33:30.356 INFO:teuthology.orchestra.run.smithi137.stdout: "complaint_time": 30, 2024-04-26T04:33:30.356 INFO:teuthology.orchestra.run.smithi137.stdout: "num_blocked_ops": 0 2024-04-26T04:33:30.357 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:30.370 DEBUG:teuthology.orchestra.run.smithi137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-26T04:33:30.418 INFO:teuthology.orchestra.run.smithi012.stderr:marked in osd.2. 2024-04-26T04:33:30.440 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-04-26T04:33:30.440 INFO:tasks.thrashosds.thrasher:inning osd 2024-04-26T04:33:30.440 INFO:tasks.thrashosds.thrasher:Adding osd 10 2024-04-26T04:33:30.440 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 10 2024-04-26T04:33:30.488 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-26T04:33:30.488 INFO:teuthology.orchestra.run.smithi137.stdout: "size": 20, 2024-04-26T04:33:30.488 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 600, 2024-04-26T04:33:30.488 INFO:teuthology.orchestra.run.smithi137.stdout: "ops": [ 2024-04-26T04:33:30.488 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.488 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:22 3.0s0 3:0061323f:::smithi17829308-8:head [append 0~131072,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:30.488 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.206381+0000", 2024-04-26T04:33:30.488 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 65.282261082000005, 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30553245699999998, 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 22 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206381+0000", 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.489 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.490 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.490 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.490 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206381+0000", 2024-04-26T04:33:30.490 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.490 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.490 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.490 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.490 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206385+0000", 2024-04-26T04:33:30.490 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3299999999999999e-06 2024-04-26T04:33:30.490 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.490 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.490 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.490 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206772+0000", 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000387673 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206797+0000", 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4156e-05 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206811+0000", 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4087000000000001e-05 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.491 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206881+0000", 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9948000000000002e-05 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206974+0000", 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.3047000000000005e-05 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207449+0000", 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00047526900000000001 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.492 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208764+0000", 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001315395 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511723+0000", 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.302958377 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511914+0000", 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000191175 2024-04-26T04:33:30.493 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.494 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.494 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.494 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.494 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.494 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:23 3.0s0 3:0061323f:::smithi17829308-8:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:30.494 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.206854+0000", 2024-04-26T04:33:30.494 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 65.281788566000003, 2024-04-26T04:33:30.494 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.407632722, 2024-04-26T04:33:30.494 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.494 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.494 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:30.494 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 23 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206854+0000", 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206854+0000", 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.495 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206855+0000", 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.133e-06 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206875+0000", 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9945999999999998e-05 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206889+0000", 2024-04-26T04:33:30.496 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.3997e-05 2024-04-26T04:33:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206899+0000", 2024-04-26T04:33:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.8490000000000001e-06 2024-04-26T04:33:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207769+0000", 2024-04-26T04:33:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00087023199999999997 2024-04-26T04:33:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.497 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:30.498 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.207895+0000", 2024-04-26T04:33:30.498 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00012591300000000001 2024-04-26T04:33:30.498 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.498 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.498 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.498 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208201+0000", 2024-04-26T04:33:30.498 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00030561 2024-04-26T04:33:30.498 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.498 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.498 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.498 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.209436+0000", 2024-04-26T04:33:30.498 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012354969999999999 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.614327+0000", 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.40489055000000002 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.614487+0000", 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00015999499999999999 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.499 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:24 3.0s0 3:0061323f:::smithi17829308-8:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.206918+0000", 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 65.281724773999997, 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.612165556, 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "started", 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 24 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.500 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206918+0000", 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206918+0000", 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206919+0000", 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.5000000000000001e-07 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.501 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.502 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.502 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206923+0000", 2024-04-26T04:33:30.502 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.0330000000000002e-06 2024-04-26T04:33:30.502 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.502 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.502 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.502 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206934+0000", 2024-04-26T04:33:30.502 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.0866999999999999e-05 2024-04-26T04:33:30.502 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.502 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.502 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.502 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.206946+0000", 2024-04-26T04:33:30.502 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.2947e-05 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208374+0000", 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0014272930000000001 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.208613+0000", 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00023916200000000001 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.503 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.615561+0000", 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.40694828199999999 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.615825+0000", 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00026369700000000002 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.819083+0000", 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20325832499999999 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.504 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=0, reqid=client.4670.0:1, at_version=32'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 65.185795760000005, 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.208159126, 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:33:30.505 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.506 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.506 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.506 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.506 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:33:30.506 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.506 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.506 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.506 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.506 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302847+0000", 2024-04-26T04:33:30.506 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.8299999999999998e-07 2024-04-26T04:33:30.506 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.506 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.506 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.507 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302898+0000", 2024-04-26T04:33:30.507 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.1421000000000001e-05 2024-04-26T04:33:30.507 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.507 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.507 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.507 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302912+0000", 2024-04-26T04:33:30.507 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4075e-05 2024-04-26T04:33:30.507 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.507 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.507 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.507 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302928+0000", 2024-04-26T04:33:30.507 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5333999999999999e-05 2024-04-26T04:33:30.507 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403201+0000", 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100273162 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403219+0000", 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7526000000000001e-05 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.510967+0000", 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107748329 2024-04-26T04:33:30.508 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.509 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.509 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.509 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511006+0000", 2024-04-26T04:33:30.509 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.8995999999999997e-05 2024-04-26T04:33:30.509 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.509 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.509 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.509 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.509 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.509 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=1, reqid=client.4670.0:2, at_version=32'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:30.509 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:33:30.509 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 65.185692682999999, 2024-04-26T04:33:30.509 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.208096686, 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.510 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.511 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.511 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.511 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302950+0000", 2024-04-26T04:33:30.511 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.82e-07 2024-04-26T04:33:30.511 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.511 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.511 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.511 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302987+0000", 2024-04-26T04:33:30.511 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.6507999999999997e-05 2024-04-26T04:33:30.511 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.511 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.511 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.511 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.302996+0000", 2024-04-26T04:33:30.512 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.4900000000000006e-06 2024-04-26T04:33:30.512 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.512 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.512 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.512 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303002+0000", 2024-04-26T04:33:30.512 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.3800000000000002e-06 2024-04-26T04:33:30.512 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.512 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.512 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.512 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403906+0000", 2024-04-26T04:33:30.512 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100904445 2024-04-26T04:33:30.512 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.512 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.403919+0000", 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.259e-05 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511012+0000", 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.107093034 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511046+0000", 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4756999999999999e-05 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.513 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 32/30 ECSubWrite(tid=2, reqid=client.4670.0:13, at_version=32'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.303011+0000", 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 65.185631219000001, 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20833385600000001, 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.514 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303011+0000", 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303011+0000", 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303012+0000", 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4700000000000002e-07 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.515 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303033+0000", 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1123000000000001e-05 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303040+0000", 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.5340000000000002e-06 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.303044+0000", 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1620000000000001e-06 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.516 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.404280+0000", 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101235978 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.404303+0000", 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2446000000000002e-05 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511267+0000", 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.1069645 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.517 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.518 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.518 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511345+0000", 2024-04-26T04:33:30.518 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.7766e-05 2024-04-26T04:33:30.518 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.518 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.518 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.518 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.518 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.518 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 32/30 ECSubWriteReply(tid=0, last_complete=32'1, committed=1, applied=1))", 2024-04-26T04:33:30.518 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.305887+0000", 2024-04-26T04:33:30.518 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 65.182755661000002, 2024-04-26T04:33:30.518 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20516263700000001, 2024-04-26T04:33:30.518 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305887+0000", 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305887+0000", 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.519 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305888+0000", 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.108e-06 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305895+0000", 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.2099999999999996e-06 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305914+0000", 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9164000000000002e-05 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.520 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.521 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.305925+0000", 2024-04-26T04:33:30.521 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1156e-05 2024-04-26T04:33:30.521 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.521 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.521 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.521 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.510931+0000", 2024-04-26T04:33:30.521 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20500514 2024-04-26T04:33:30.521 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.521 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.521 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.521 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511049+0000", 2024-04-26T04:33:30.521 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00011885899999999999 2024-04-26T04:33:30.521 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.522 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.522 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.522 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.522 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.522 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 32/30 ECSubWriteReply(tid=1, last_complete=32'2, committed=1, applied=1))", 2024-04-26T04:33:30.522 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:25.306725+0000", 2024-04-26T04:33:30.522 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 65.181917295000005, 2024-04-26T04:33:30.522 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20445080600000001, 2024-04-26T04:33:30.522 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.522 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.522 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.522 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.522 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306725+0000", 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306725+0000", 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306726+0000", 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.2500000000000004e-07 2024-04-26T04:33:30.523 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.524 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.524 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.524 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306731+0000", 2024-04-26T04:33:30.524 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.9250000000000003e-06 2024-04-26T04:33:30.524 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.524 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.524 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.524 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306745+0000", 2024-04-26T04:33:30.524 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.4367e-05 2024-04-26T04:33:30.524 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.524 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.306753+0000", 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.9699999999999999e-06 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511144+0000", 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20439093 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:25.511176+0000", 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.1689000000000001e-05 2024-04-26T04:33:30.525 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:280 3.0s0 3:0061323f:::smithi17829308-8:head [delete] snapc 2=[2] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:30.865956+0000", 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 59.622686328, 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20424141400000001, 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:33:30.526 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 280 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865956+0000", 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865956+0000", 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.527 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865959+0000", 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0570000000000001e-06 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.865971+0000", 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.1576000000000001e-05 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866003+0000", 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2543999999999999e-05 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.528 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866020+0000", 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6433e-05 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866157+0000", 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013767200000000001 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.866327+0000", 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00016995199999999999 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.529 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.530 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.530 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.867476+0000", 2024-04-26T04:33:30.530 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0011486700000000001 2024-04-26T04:33:30.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.530 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.530 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.530 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:30.869167+0000", 2024-04-26T04:33:30.530 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016912450000000001 2024-04-26T04:33:30.530 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.530 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.530 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:30.530 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.070080+0000", 2024-04-26T04:33:30.530 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200912916 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:31.070198+0000", 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000117349 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds2 40/30 ECSubWrite(tid=33, reqid=client.4670.0:332, at_version=0'0, trim_to=0'0, roll_forward_to=40'19))", 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:35.023904+0000", 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 55.464738105999999, 2024-04-26T04:33:30.531 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201717813, 2024-04-26T04:33:30.532 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.532 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.532 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.532 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.532 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.532 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.532 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023904+0000", 2024-04-26T04:33:30.532 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.532 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023904+0000", 2024-04-26T04:33:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023906+0000", 2024-04-26T04:33:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.387e-06 2024-04-26T04:33:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.533 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023925+0000", 2024-04-26T04:33:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9059000000000002e-05 2024-04-26T04:33:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023952+0000", 2024-04-26T04:33:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.7455999999999999e-05 2024-04-26T04:33:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.023968+0000", 2024-04-26T04:33:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.526e-05 2024-04-26T04:33:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.534 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.224267+0000", 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200299636 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.224282+0000", 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5007e-05 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.225547+0000", 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012650719999999999 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.535 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:35.225622+0000", 2024-04-26T04:33:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 7.4936000000000002e-05 2024-04-26T04:33:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.536 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.as4 62/30 ECSubWrite(tid=49, reqid=client.4670.0:645, at_version=0'0, trim_to=0'0, roll_forward_to=62'36))", 2024-04-26T04:33:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:32:57.714367+0000", 2024-04-26T04:33:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 32.774275670000002, 2024-04-26T04:33:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201555649, 2024-04-26T04:33:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.537 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.538 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.714367+0000", 2024-04-26T04:33:30.538 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.538 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.538 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.538 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.714367+0000", 2024-04-26T04:33:30.538 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.538 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.538 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.538 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.714369+0000", 2024-04-26T04:33:30.538 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3199999999999998e-06 2024-04-26T04:33:30.538 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.539 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.539 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.539 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.714392+0000", 2024-04-26T04:33:30.539 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3249000000000001e-05 2024-04-26T04:33:30.539 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.539 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.539 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.539 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.714437+0000", 2024-04-26T04:33:30.539 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.4203999999999999e-05 2024-04-26T04:33:30.539 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.539 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.539 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.540 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.714460+0000", 2024-04-26T04:33:30.540 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.3163000000000002e-05 2024-04-26T04:33:30.540 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.540 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.540 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.540 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.914817+0000", 2024-04-26T04:33:30.540 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20035745399999999 2024-04-26T04:33:30.540 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.540 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.540 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.540 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.914835+0000", 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7839e-05 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.915888+0000", 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0010526979999999999 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:32:57.915922+0000", 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.4721999999999997e-05 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.541 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.542 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.542 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:722 3.9s0 3:979b0971:::smithi17829308-28:head [stat,delete] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-26T04:33:30.542 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:01.995106+0000", 2024-04-26T04:33:30.542 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 28.493536807000002, 2024-04-26T04:33:30.542 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.30568222699999997, 2024-04-26T04:33:30.542 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.542 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.542 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:30.542 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:33:30.543 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:33:30.543 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:30.543 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 722 2024-04-26T04:33:30.543 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.543 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.543 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.543 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.543 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995106+0000", 2024-04-26T04:33:30.543 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995106+0000", 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995108+0000", 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.6070000000000002e-06 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.544 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995121+0000", 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.272e-05 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995155+0000", 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3639000000000001e-05 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995171+0000", 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.6427e-05 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.545 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.546 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995367+0000", 2024-04-26T04:33:30.546 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00019574499999999999 2024-04-26T04:33:30.546 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.546 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.546 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:30.546 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.995540+0000", 2024-04-26T04:33:30.546 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017338799999999999 2024-04-26T04:33:30.546 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.546 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.546 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.546 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.997167+0000", 2024-04-26T04:33:30.546 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0016268700000000001 2024-04-26T04:33:30.547 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.547 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.547 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.547 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:01.999322+0000", 2024-04-26T04:33:30.547 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0021550810000000001 2024-04-26T04:33:30.547 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.547 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.547 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:30.547 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.300652+0000", 2024-04-26T04:33:30.547 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.301329601 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.300788+0000", 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00013614899999999999 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.as4 66/30 ECSubWrite(tid=53, reqid=client.4670.0:702, at_version=0'0, trim_to=0'0, roll_forward_to=66'39))", 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:02.315960+0000", 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 28.172682812000001, 2024-04-26T04:33:30.548 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201565358, 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.315960+0000", 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.315960+0000", 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.549 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.315962+0000", 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1639999999999999e-06 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.315988+0000", 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5868999999999999e-05 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.316053+0000", 2024-04-26T04:33:30.550 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.4832999999999994e-05 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.416155+0000", 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100102605 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.516408+0000", 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10025274000000001 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.551 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.516425+0000", 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7283e-05 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.517433+0000", 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0010080779999999999 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.517525+0000", 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.1786000000000005e-05 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.552 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 67/30 ECSubWriteReply(tid=41, last_complete=66'8, committed=1, applied=1))", 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:02.402204+0000", 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 28.086438639000001, 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.300648372, 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.402204+0000", 2024-04-26T04:33:30.553 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.554 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.554 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.554 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.402204+0000", 2024-04-26T04:33:30.554 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.554 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.554 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.554 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.402206+0000", 2024-04-26T04:33:30.554 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.703e-06 2024-04-26T04:33:30.554 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.554 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.402215+0000", 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 9.6500000000000008e-06 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.402244+0000", 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9156999999999999e-05 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.502387+0000", 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10014247 2024-04-26T04:33:30.555 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.702789+0000", 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.200401736 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:02.702852+0000", 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.3656000000000002e-05 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.556 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 68/30 ECSubWrite(tid=107, reqid=osd.5.0:107, at_version=68'81, trim_to=0'0, roll_forward_to=61'79))", 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.433160+0000", 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 27.055482567999999, 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20302263600000001, 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.433160+0000", 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.557 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.433160+0000", 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.433160+0000", 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.2599999999999998e-07 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.433208+0000", 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.7320000000000001e-05 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.558 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.433232+0000", 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.4620000000000001e-05 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.433259+0000", 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.656e-05 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.533592+0000", 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10033310300000001 2024-04-26T04:33:30.559 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.533607+0000", 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5330000000000001e-05 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.636100+0000", 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.102492578 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.636183+0000", 2024-04-26T04:33:30.560 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.2699000000000002e-05 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.ds2 68/30 ECSubWrite(tid=72, reqid=osd.7.0:72, at_version=68'50, trim_to=0'0, roll_forward_to=67'46))", 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.705256+0000", 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.783386700000001, 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20317829500000001, 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.561 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.562 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.562 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705256+0000", 2024-04-26T04:33:30.562 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.562 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.562 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.562 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.562 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705256+0000", 2024-04-26T04:33:30.562 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.562 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.562 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.562 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.562 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705257+0000", 2024-04-26T04:33:30.562 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.424e-06 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705304+0000", 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.6879000000000003e-05 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705339+0000", 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.472e-05 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.563 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.705364+0000", 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.5360000000000001e-05 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.905749+0000", 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20038445199999999 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.905766+0000", 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.698e-05 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.564 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.908334+0000", 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0025682439999999999 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.908434+0000", 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00010023599999999999 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 68/30 ECSubWrite(tid=110, reqid=osd.5.0:110, at_version=68'85, trim_to=0'0, roll_forward_to=68'83))", 2024-04-26T04:33:30.565 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:03.838699+0000", 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 26.649943087, 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.201654738, 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.838699+0000", 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.566 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.838699+0000", 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.838701+0000", 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.9630000000000001e-06 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.838743+0000", 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 4.1579000000000003e-05 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.567 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.838772+0000", 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.9507000000000001e-05 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:03.938855+0000", 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.10008270900000001 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.039044+0000", 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100188612 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.568 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.039062+0000", 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.8085999999999999e-05 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.040266+0000", 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0012044759999999999 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:04.040354+0000", 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 8.7806000000000006e-05 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.569 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 84/30 ECSubWrite(tid=150, reqid=client.4670.0:933, at_version=84'133, trim_to=0'0, roll_forward_to=83'126))", 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:19.304868+0000", 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 11.183774269000001, 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.25729703399999998, 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.570 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.304868+0000", 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.304868+0000", 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.304870+0000", 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.736e-06 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.571 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.305070+0000", 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00020017900000000001 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.305101+0000", 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.0692999999999999e-05 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.405190+0000", 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100089434 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.572 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.460880+0000", 2024-04-26T04:33:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.055689331000000002 2024-04-26T04:33:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.460902+0000", 2024-04-26T04:33:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.2847e-05 2024-04-26T04:33:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.562111+0000", 2024-04-26T04:33:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.101208585 2024-04-26T04:33:30.573 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.562165+0000", 2024-04-26T04:33:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 5.4228999999999999e-05 2024-04-26T04:33:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.575 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "osd_op(client.4670.0:939 3.0s0 3:0061323f:::smithi17829308-8:head [create,setxattr qe_tlRrqOVZN]Sg`OogokNcdi`iTe[kSpgXo[tdi (347) in=40b,setxattr s_\\R^hs]`PhSfjan[pki[NgTNmQrWYik__[]RgPU (594) in=40b,setxattr `RVkQO\\NPp`^id``ktR\\nNYoo[aZVa[a (1401) in=32b,setxattr eUkd\\WaahPacqTYl^iV[kQsis\\aXTQqklkRsYTbs (873) in=40b,setxattr q\\qjarW`i[`bSljr`srl\\Vqni]Z`STgbdTnSmh`o (1963) in=40b,setxattr Sd^`oRTnVWgjdbTpljaQYrhO[``]a^haiQjgUoq` (18032) in=40b,setxattr nUtsh_fZqO^tgbj\\TdjN\\iNYcaa[UnfnknanjssP (1087) in=40b,setxattr _kcTXgOY]Up]^Vac[sbWiZ_aZifkbn`Vphdsslam (1434) in=40b,setxattr ttpkrgNZdj[]rR]iWqO]TaqdilgcW_tR\\^[_Pmgi (1083) in=40b,setxattr gYWdpfi^\\XWiW\\OjsQl_iOetb_TPV\\saa`i]Zkab (593) in=40b,setxattr qijT`_VpNN[TW`kts_Y_ngsTQrrOe_YNONO[^Qla (19224) in=40b,setxattr jXkpPeq`\\r\\hVgrkgqYiY_s[Up`[etXec^VWjblQ (175) in=40b,setxattr Ohgbf\\VQaSpfn^apdUcdOWhk]WUmXU\\`_NcacXQt (139) in=40b,setxattr [egW_laW[O\\dpqaiSbfc]Za^tico``qipqj`Pte` (1050) in=40b,setxattr h\\kdtVlaQPNaZakj[Y`XiYgiflaRcnt\\qnpaNURY (981) in=40b,setxattr as^bS\\_VZOPUeOaf\\]dtfgretlb]]`s``POt_SXc (19508) in=40b,setxattr NQg\\ajVioao__Weae_To_SsOrScUoVTbZRm[oc[O (1297) in=40b,setxattr UZTkha_tgerT`qq\\omSTjZSSXXZQU^f[RWr]`Znn (522) in=40b,setxattr dsdNaS_PtcaUjlnr]mXhVnPO_aZl]RWdh (1401) in=33b,setxattr pNnNhSVlYtVaX`q^i[ZgRflmt\\V_ZpabhitfmqXf (31) in=40b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:19.324910+0000", 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 11.1637328, 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.40532692199999998, 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: "client_info": { 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: "client": "client.4670", 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: "tid": 939 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.576 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.324910+0000", 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.324910+0000", 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.324913+0000", 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.3440000000000001e-06 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.577 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.325091+0000", 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00017758199999999999 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.325123+0000", 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2829e-05 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.325139+0000", 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.5809999999999999e-05 2024-04-26T04:33:30.578 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.354688+0000", 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.029548334999999998 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "started", 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.354871+0000", 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.00018355499999999999 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.356542+0000", 2024-04-26T04:33:30.579 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.001671259 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.359491+0000", 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0029483280000000001 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "commit_sent", 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.729973+0000", 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.37048254899999999 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.580 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:19.730237+0000", 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.000263331 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: "description": "MOSDECSubOpWrite(3.8s4 89/30 ECSubWrite(tid=165, reqid=osd.5.0:164, at_version=0'0, trim_to=0'0, roll_forward_to=89'144))", 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: "initiated_at": "2024-04-26T04:33:24.756669+0000", 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: "age": 5.7319731420000002, 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.20258427700000001, 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: "continuous": false, 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: "type_data": { 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: "flag_point": "reached pg", 2024-04-26T04:33:30.581 INFO:teuthology.orchestra.run.smithi137.stdout: "events": [ 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "initiated", 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.756669+0000", 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "header_read", 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.756669+0000", 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "throttled", 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.756671+0000", 2024-04-26T04:33:30.582 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.7400000000000001e-06 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "all_read", 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.756692+0000", 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 2.1098999999999999e-05 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "dispatched", 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.756725+0000", 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 3.2639999999999999e-05 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.856867+0000", 2024-04-26T04:33:30.583 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100142332 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "reached_pg", 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.957162+0000", 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.100294381 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_started", 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.957177+0000", 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 1.523e-05 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "sub_op_committed", 2024-04-26T04:33:30.584 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.959184+0000", 2024-04-26T04:33:30.585 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 0.0020072060000000001 2024-04-26T04:33:30.585 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-26T04:33:30.585 INFO:teuthology.orchestra.run.smithi137.stdout: { 2024-04-26T04:33:30.585 INFO:teuthology.orchestra.run.smithi137.stdout: "event": "done", 2024-04-26T04:33:30.585 INFO:teuthology.orchestra.run.smithi137.stdout: "time": "2024-04-26T04:33:24.959254+0000", 2024-04-26T04:33:30.585 INFO:teuthology.orchestra.run.smithi137.stdout: "duration": 6.9648999999999997e-05 2024-04-26T04:33:30.585 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.585 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.585 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.585 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-26T04:33:30.585 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-26T04:33:30.585 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-26T04:33:30.588 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-26T04:33:30.722 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:30.722 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:30.722 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 0 2024-04-26T04:33:30.722 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:30.736 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-26T04:33:30.750 INFO:tasks.ceph.mon.a.smithi012.stderr:2024-04-26T04:33:30.745+0000 7f05ec032640 -1 mon.a@0(leader).osd e95 definitely_dead 0 2024-04-26T04:33:30.854 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:30.854 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:30.854 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:33:30.854 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:33:30.854 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:30.869 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-26T04:33:30.995 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:30.995 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:33:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:33:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:265 3.as0 3:51a91c23:::smithi17829308-18:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-26T04:33:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:29.817865+0000", 2024-04-26T04:33:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 61.174084207999996, 2024-04-26T04:33:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50897067299999998, 2024-04-26T04:33:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:30.996 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 265 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817865+0000", 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817865+0000", 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:30.997 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:30.998 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:30.998 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817867+0000", 2024-04-26T04:33:30.998 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0999999999999998e-06 2024-04-26T04:33:30.998 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:30.998 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:30.998 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:30.998 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817876+0000", 2024-04-26T04:33:30.998 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8060000000000001e-06 2024-04-26T04:33:30.998 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:30.998 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:30.998 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:30.998 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817899+0000", 2024-04-26T04:33:30.998 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2716000000000001e-05 2024-04-26T04:33:30.999 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:30.999 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:30.999 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:30.999 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.817910+0000", 2024-04-26T04:33:30.999 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1579999999999999e-05 2024-04-26T04:33:30.999 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:30.999 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:30.999 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.822433+0000", 2024-04-26T04:33:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0045222869999999998 2024-04-26T04:33:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.822747+0000", 2024-04-26T04:33:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00031401299999999998 2024-04-26T04:33:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.823790+0000", 2024-04-26T04:33:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00104357 2024-04-26T04:33:31.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:29.924740+0000", 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100949605 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.326718+0000", 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40197800900000002 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:30.326836+0000", 2024-04-26T04:33:31.001 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011798699999999999 2024-04-26T04:33:31.002 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.002 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.002 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.002 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.002 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.002 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:329 3.as0 3:527650bd:::smithi17829308-27:head [rollback 2] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:31.002 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.408276+0000", 2024-04-26T04:33:31.002 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 56.583672972000002, 2024-04-26T04:33:31.002 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61054785099999997, 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 329 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408276+0000", 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.003 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408276+0000", 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408277+0000", 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.7999999999999993e-07 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408284+0000", 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.0389999999999999e-06 2024-04-26T04:33:31.004 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408311+0000", 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6452999999999999e-05 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408324+0000", 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3259e-05 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408439+0000", 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000114789 2024-04-26T04:33:31.005 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.006 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.006 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:31.006 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408444+0000", 2024-04-26T04:33:31.006 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.3700000000000003e-06 2024-04-26T04:33:31.006 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.006 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.006 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.006 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.414140+0000", 2024-04-26T04:33:31.006 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0056952110000000004 2024-04-26T04:33:31.006 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.006 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.006 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.007 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.414263+0000", 2024-04-26T04:33:31.007 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012392699999999999 2024-04-26T04:33:31.007 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.007 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.007 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.007 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.415016+0000", 2024-04-26T04:33:31.007 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00075301600000000002 2024-04-26T04:33:31.007 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.007 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.007 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.616959+0000", 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20194278800000001 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018750+0000", 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40179092999999999 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018824+0000", 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.4089000000000002e-05 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.008 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.009 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.009 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.009 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:330 3.as0 3:527650bd:::smithi17829308-27:head [append 0~0] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:31.009 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.408341+0000", 2024-04-26T04:33:31.009 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 56.583608138000002, 2024-04-26T04:33:31.009 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61070586699999996, 2024-04-26T04:33:31.009 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.009 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.009 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.009 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.010 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.010 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.010 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 330 2024-04-26T04:33:31.010 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.010 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.010 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.010 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.010 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408341+0000", 2024-04-26T04:33:31.010 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.010 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.011 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.011 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.011 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408341+0000", 2024-04-26T04:33:31.011 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.011 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.011 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.011 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.011 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408342+0000", 2024-04-26T04:33:31.011 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.6700000000000003e-07 2024-04-26T04:33:31.011 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.012 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.012 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.012 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408346+0000", 2024-04-26T04:33:31.012 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7299999999999999e-06 2024-04-26T04:33:31.012 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.012 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.012 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.012 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408357+0000", 2024-04-26T04:33:31.012 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1276e-05 2024-04-26T04:33:31.012 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.012 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.013 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.013 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408371+0000", 2024-04-26T04:33:31.013 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3763e-05 2024-04-26T04:33:31.013 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.013 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.013 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.013 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408472+0000", 2024-04-26T04:33:31.013 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010149600000000001 2024-04-26T04:33:31.013 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.013 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.013 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:31.014 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.408477+0000", 2024-04-26T04:33:31.014 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5029999999999999e-06 2024-04-26T04:33:31.014 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.014 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.014 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.014 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.515978+0000", 2024-04-26T04:33:31.014 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.107501546 2024-04-26T04:33:31.014 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.014 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.014 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.014 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.516082+0000", 2024-04-26T04:33:31.014 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010331 2024-04-26T04:33:31.014 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.516573+0000", 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000491316 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.617004+0000", 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100431009 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018996+0000", 2024-04-26T04:33:31.015 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40199249199999998 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.019047+0000", 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0658999999999997e-05 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.520051+0000", 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 56.471898013000001, 2024-04-26T04:33:31.016 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49885426900000002, 2024-04-26T04:33:31.017 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.017 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.017 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:31.017 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.017 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.017 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.017 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520051+0000", 2024-04-26T04:33:31.017 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.017 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.017 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.017 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.017 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520051+0000", 2024-04-26T04:33:31.018 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.018 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.018 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.018 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520052+0000", 2024-04-26T04:33:31.018 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.5000000000000001e-07 2024-04-26T04:33:31.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.018 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.018 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.018 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520062+0000", 2024-04-26T04:33:31.018 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9799999999999993e-06 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520079+0000", 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6518999999999999e-05 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.520087+0000", 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.5329999999999994e-06 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.019 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018885+0000", 2024-04-26T04:33:31.020 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49879782299999997 2024-04-26T04:33:31.020 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.020 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.020 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.020 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018906+0000", 2024-04-26T04:33:31.020 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0564000000000002e-05 2024-04-26T04:33:31.020 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.020 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.as0 40/30 ECSubWriteReply(tid=28, last_complete=40'20, committed=1, applied=1))", 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:34.522047+0000", 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 56.469902328000003, 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49659926700000001, 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.021 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522047+0000", 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522047+0000", 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522048+0000", 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1200000000000001e-07 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.022 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.023 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522054+0000", 2024-04-26T04:33:31.023 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.049e-06 2024-04-26T04:33:31.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.023 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.023 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.023 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522070+0000", 2024-04-26T04:33:31.023 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5492999999999999e-05 2024-04-26T04:33:31.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.023 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.023 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.024 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:34.522079+0000", 2024-04-26T04:33:31.024 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1679999999999993e-06 2024-04-26T04:33:31.024 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.024 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.024 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.024 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018623+0000", 2024-04-26T04:33:31.024 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49654389700000001 2024-04-26T04:33:31.024 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.024 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.024 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.024 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.018646+0000", 2024-04-26T04:33:31.024 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3748e-05 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:333 3.as0 3:527650bd:::smithi17829308-27:head [copy-from ver 2] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.120049+0000", 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 55.871900594000003, 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.66333988200000005, 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.025 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.026 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.026 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 333 2024-04-26T04:33:31.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.026 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.026 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.026 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.026 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120049+0000", 2024-04-26T04:33:31.026 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.026 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.026 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.026 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120049+0000", 2024-04-26T04:33:31.026 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120050+0000", 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.536e-06 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120057+0000", 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6869999999999997e-06 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.027 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120074+0000", 2024-04-26T04:33:31.028 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6787e-05 2024-04-26T04:33:31.028 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.028 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.028 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.028 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120086+0000", 2024-04-26T04:33:31.028 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2665e-05 2024-04-26T04:33:31.028 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.028 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.029 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.029 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.220306+0000", 2024-04-26T04:33:31.029 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10021996900000001 2024-04-26T04:33:31.029 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.029 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.029 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.029 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.220485+0000", 2024-04-26T04:33:31.029 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017836800000000001 2024-04-26T04:33:31.029 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.029 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.029 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.030 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.577412+0000", 2024-04-26T04:33:31.030 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.35692740299999998 2024-04-26T04:33:31.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.030 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.030 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.030 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.580481+0000", 2024-04-26T04:33:31.030 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.003068735 2024-04-26T04:33:31.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.031 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.031 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.031 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.783160+0000", 2024-04-26T04:33:31.031 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20267940200000001 2024-04-26T04:33:31.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.031 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.031 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.031 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.783389+0000", 2024-04-26T04:33:31.031 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022833 2024-04-26T04:33:31.031 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.031 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.031 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:334 3.as0 3:527650bd:::smithi17829308-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.120106+0000", 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 55.871843110999997, 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.459292215, 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 334 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.032 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120106+0000", 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120106+0000", 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120107+0000", 2024-04-26T04:33:31.033 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.540000000000001e-07 2024-04-26T04:33:31.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.034 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.034 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.034 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120111+0000", 2024-04-26T04:33:31.034 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.6160000000000002e-06 2024-04-26T04:33:31.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.034 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.034 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.035 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.120124+0000", 2024-04-26T04:33:31.035 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2965e-05 2024-04-26T04:33:31.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.035 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.035 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.220210+0000", 2024-04-26T04:33:31.035 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100086047 2024-04-26T04:33:31.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.036 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.036 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.036 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221197+0000", 2024-04-26T04:33:31.036 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00098664200000000003 2024-04-26T04:33:31.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.036 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.036 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:31.036 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.221256+0000", 2024-04-26T04:33:31.037 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9734000000000001e-05 2024-04-26T04:33:31.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.037 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.037 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.578868+0000", 2024-04-26T04:33:31.037 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.35761166799999999 2024-04-26T04:33:31.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.038 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.038 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.038 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579093+0000", 2024-04-26T04:33:31.038 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022499099999999999 2024-04-26T04:33:31.038 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.039 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.039 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579399+0000", 2024-04-26T04:33:31.039 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00030559800000000002 2024-04-26T04:33:31.039 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.039 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.039 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.040 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:620 3.as0 3:57b6ca82:::smithi17829308-14:b [read 0~229376,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:31.040 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:55.807425+0000", 2024-04-26T04:33:31.040 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 35.184524615999997, 2024-04-26T04:33:31.040 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50556072699999999, 2024-04-26T04:33:31.040 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.041 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.041 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.041 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.042 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.042 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.042 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 620 2024-04-26T04:33:31.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.042 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.043 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.043 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.043 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807425+0000", 2024-04-26T04:33:31.043 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.044 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.044 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807425+0000", 2024-04-26T04:33:31.044 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.045 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.045 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.045 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807429+0000", 2024-04-26T04:33:31.045 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.0459999999999999e-06 2024-04-26T04:33:31.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.045 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.045 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.046 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807448+0000", 2024-04-26T04:33:31.046 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9553000000000001e-05 2024-04-26T04:33:31.046 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.046 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.046 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807498+0000", 2024-04-26T04:33:31.046 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.9369000000000001e-05 2024-04-26T04:33:31.046 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807524+0000", 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5843e-05 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907865+0000", 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100341135 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.908044+0000", 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017972100000000001 2024-04-26T04:33:31.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.312986+0000", 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40494106000000002 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:621 3.as0 3:57b6ca82:::smithi17829308-14:b [read 0~229376,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:55.807584+0000", 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 35.184365268999997, 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50635941100000004, 2024-04-26T04:33:31.048 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.049 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.049 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.049 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.049 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.049 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.049 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 621 2024-04-26T04:33:31.049 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.049 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.050 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.050 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.050 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807584+0000", 2024-04-26T04:33:31.050 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.050 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.050 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.050 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.051 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807584+0000", 2024-04-26T04:33:31.051 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.051 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.051 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807586+0000", 2024-04-26T04:33:31.051 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.466e-06 2024-04-26T04:33:31.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.051 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.052 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807595+0000", 2024-04-26T04:33:31.052 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.064e-06 2024-04-26T04:33:31.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.052 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.052 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.052 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.807616+0000", 2024-04-26T04:33:31.052 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1379e-05 2024-04-26T04:33:31.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.052 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.052 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.052 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907810+0000", 2024-04-26T04:33:31.052 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10019436800000001 2024-04-26T04:33:31.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.009320+0000", 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101509213 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.009472+0000", 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015200399999999999 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.313944+0000", 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30447191699999998 2024-04-26T04:33:31.053 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.054 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.054 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.054 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.054 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.054 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:622 3.as0 3:57b6ca82:::smithi17829308-14:b [read 0~229376,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:31.054 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:55.907855+0000", 2024-04-26T04:33:31.054 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 35.084094489999998, 2024-04-26T04:33:31.054 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.806439982, 2024-04-26T04:33:31.054 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.054 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.054 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.054 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.054 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.055 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.055 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 622 2024-04-26T04:33:31.055 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.055 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.055 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.055 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.055 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907855+0000", 2024-04-26T04:33:31.055 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.055 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.056 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.056 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.056 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907855+0000", 2024-04-26T04:33:31.056 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.056 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.056 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.056 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.056 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907857+0000", 2024-04-26T04:33:31.056 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2000000000000001e-06 2024-04-26T04:33:31.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.057 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.057 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.057 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907869+0000", 2024-04-26T04:33:31.057 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.19e-05 2024-04-26T04:33:31.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.057 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.057 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.057 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907889+0000", 2024-04-26T04:33:31.058 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9643000000000001e-05 2024-04-26T04:33:31.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.058 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.058 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.058 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:55.907901+0000", 2024-04-26T04:33:31.058 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2289999999999999e-05 2024-04-26T04:33:31.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.059 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.059 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.210837+0000", 2024-04-26T04:33:31.059 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30293618300000003 2024-04-26T04:33:31.059 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.059 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.059 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.211009+0000", 2024-04-26T04:33:31.059 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017150999999999999 2024-04-26T04:33:31.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.060 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.060 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.060 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.714295+0000", 2024-04-26T04:33:31.060 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50328625599999999 2024-04-26T04:33:31.060 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.060 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.060 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.061 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.061 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:645 3.as0 3:51a91c23:::smithi17829308-18:head [copy-from ver 49] snapc f=[f,e,b] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-26T04:33:31.061 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:57.194710+0000", 2024-04-26T04:33:31.061 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 33.797239799000003, 2024-04-26T04:33:31.061 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.517523497, 2024-04-26T04:33:31.061 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.061 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.062 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.062 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.062 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.063 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.063 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 645 2024-04-26T04:33:31.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.063 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.063 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.063 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194710+0000", 2024-04-26T04:33:31.063 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.064 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.064 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194710+0000", 2024-04-26T04:33:31.064 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.064 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.064 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194714+0000", 2024-04-26T04:33:31.065 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.07e-06 2024-04-26T04:33:31.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.065 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.065 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.065 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194727+0000", 2024-04-26T04:33:31.065 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3593e-05 2024-04-26T04:33:31.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.065 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.065 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.066 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194762+0000", 2024-04-26T04:33:31.066 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.5086000000000003e-05 2024-04-26T04:33:31.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.066 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.066 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.066 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194782+0000", 2024-04-26T04:33:31.066 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9349000000000001e-05 2024-04-26T04:33:31.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.067 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.067 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.067 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194905+0000", 2024-04-26T04:33:31.067 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000123497 2024-04-26T04:33:31.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.067 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.068 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:31.068 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.194923+0000", 2024-04-26T04:33:31.068 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7377e-05 2024-04-26T04:33:31.068 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.068 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.068 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.068 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.298801+0000", 2024-04-26T04:33:31.068 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10387803 2024-04-26T04:33:31.068 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.069 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.069 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.069 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.298959+0000", 2024-04-26T04:33:31.069 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000158484 2024-04-26T04:33:31.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.069 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.069 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.069 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.509187+0000", 2024-04-26T04:33:31.069 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21022799 2024-04-26T04:33:31.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.070 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.070 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.611520+0000", 2024-04-26T04:33:31.070 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.102332901 2024-04-26T04:33:31.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.070 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.071 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.712044+0000", 2024-04-26T04:33:31.071 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10052366 2024-04-26T04:33:31.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.071 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.071 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:57.712233+0000", 2024-04-26T04:33:31.071 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018945999999999999 2024-04-26T04:33:31.071 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.071 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.071 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.072 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:702 3.as0 3:527650bd:::smithi17829308-27:head [copy-from ver 44] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-26T04:33:31.072 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:01.580554+0000", 2024-04-26T04:33:31.072 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 29.411394905000002, 2024-04-26T04:33:31.072 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.73333148100000001, 2024-04-26T04:33:31.072 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.072 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.072 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.072 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.073 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.073 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.073 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 702 2024-04-26T04:33:31.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.073 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.073 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.073 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.073 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580554+0000", 2024-04-26T04:33:31.073 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.073 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.073 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.074 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580554+0000", 2024-04-26T04:33:31.074 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.074 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.074 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.074 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580556+0000", 2024-04-26T04:33:31.074 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9910000000000001e-06 2024-04-26T04:33:31.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.074 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.074 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.074 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580570+0000", 2024-04-26T04:33:31.074 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3798000000000001e-05 2024-04-26T04:33:31.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580599+0000", 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8591000000000001e-05 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580615+0000", 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6515e-05 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580817+0000", 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00020131200000000001 2024-04-26T04:33:31.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.076 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.076 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.581112+0000", 2024-04-26T04:33:31.076 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00029485500000000001 2024-04-26T04:33:31.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.076 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.076 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.109360+0000", 2024-04-26T04:33:31.076 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.52824810099999997 2024-04-26T04:33:31.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.077 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.077 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.114011+0000", 2024-04-26T04:33:31.077 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0046516190000000001 2024-04-26T04:33:31.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.077 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.077 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.313710+0000", 2024-04-26T04:33:31.077 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.19969835499999999 2024-04-26T04:33:31.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.077 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.078 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.313886+0000", 2024-04-26T04:33:31.078 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017634399999999999 2024-04-26T04:33:31.078 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.078 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.078 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.078 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:703 3.as0 3:527650bd:::smithi17829308-27:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e66)", 2024-04-26T04:33:31.078 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:01.580660+0000", 2024-04-26T04:33:31.078 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 29.411289105000002, 2024-04-26T04:33:31.078 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.53310025599999999, 2024-04-26T04:33:31.078 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.079 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.079 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.079 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.079 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.079 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.079 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 703 2024-04-26T04:33:31.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.079 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.079 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.079 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580660+0000", 2024-04-26T04:33:31.079 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580660+0000", 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580661+0000", 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.95e-07 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580666+0000", 2024-04-26T04:33:31.080 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0919999999999997e-06 2024-04-26T04:33:31.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.081 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.081 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580684+0000", 2024-04-26T04:33:31.081 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7578e-05 2024-04-26T04:33:31.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.081 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.081 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.580694+0000", 2024-04-26T04:33:31.082 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0339999999999999e-05 2024-04-26T04:33:31.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.082 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.082 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.781619+0000", 2024-04-26T04:33:31.082 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20092432399999999 2024-04-26T04:33:31.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.082 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:31.083 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:01.781728+0000", 2024-04-26T04:33:31.083 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00010893200000000001 2024-04-26T04:33:31.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.083 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.083 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.113439+0000", 2024-04-26T04:33:31.083 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.33171105400000001 2024-04-26T04:33:31.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.083 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.083 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.113592+0000", 2024-04-26T04:33:31.083 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000153731 2024-04-26T04:33:31.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.113761+0000", 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016820999999999999 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:825 3.as0 3:51a91c23:::smithi17829308-18:head [read 0~393216,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.293542+0000", 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 23.698407706000001, 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.42480815300000002, 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.084 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 825 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293542+0000", 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.085 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293542+0000", 2024-04-26T04:33:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293542+0000", 2024-04-26T04:33:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.2500000000000001e-07 2024-04-26T04:33:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293545+0000", 2024-04-26T04:33:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3529999999999999e-06 2024-04-26T04:33:31.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293552+0000", 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.5570000000000001e-06 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293559+0000", 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1420000000000004e-06 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293779+0000", 2024-04-26T04:33:31.087 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022003100000000001 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.293797+0000", 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8102999999999999e-05 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.398220+0000", 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10442280600000001 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.088 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.398489+0000", 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026918299999999999 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.718350+0000", 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.31986055299999999 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:836 3.as0 3:57b6ca82:::smithi17829308-14:head [rollback 14] snapc 16=[16,15,14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:09.269087+0000", 2024-04-26T04:33:31.089 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 21.722862265, 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0499148229999999, 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 836 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269087+0000", 2024-04-26T04:33:31.090 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.091 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.091 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269087+0000", 2024-04-26T04:33:31.091 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.091 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.091 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269089+0000", 2024-04-26T04:33:31.091 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4470000000000001e-06 2024-04-26T04:33:31.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.091 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269099+0000", 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0657e-05 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269120+0000", 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1222000000000001e-05 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269137+0000", 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6143000000000001e-05 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.092 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.093 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.675959+0000", 2024-04-26T04:33:31.093 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40682288 2024-04-26T04:33:31.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.093 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for peered", 2024-04-26T04:33:31.093 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.675965+0000", 2024-04-26T04:33:31.093 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4559999999999999e-06 2024-04-26T04:33:31.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.093 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.093 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:11.102281+0000", 2024-04-26T04:33:31.093 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4263155759999999 2024-04-26T04:33:31.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for missing object", 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:11.102551+0000", 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000270396 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.109278+0000", 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.006727003 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.109514+0000", 2024-04-26T04:33:31.094 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00023647699999999999 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.110889+0000", 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0013741630000000001 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.114040+0000", 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0031512649999999999 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.318345+0000", 2024-04-26T04:33:31.095 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20430549100000001 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.319002+0000", 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000656647 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:837 3.as0 3:57b6ca82:::smithi17829308-14:head [append 0~0] snapc 16=[16,15,14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:09.269156+0000", 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 21.722793072000002, 2024-04-26T04:33:31.096 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1507874779999998, 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 837 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269156+0000", 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269156+0000", 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269158+0000", 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2160000000000001e-06 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269163+0000", 2024-04-26T04:33:31.098 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.5770000000000001e-06 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269181+0000", 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7629999999999999e-05 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269198+0000", 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7376000000000001e-05 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.676017+0000", 2024-04-26T04:33:31.099 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40681856900000002 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for peered", 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.676018+0000", 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.59e-06 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:11.102957+0000", 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4269388190000001 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for missing object", 2024-04-26T04:33:31.100 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:11.103123+0000", 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000165909 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.111738+0000", 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0086147130000001 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.111986+0000", 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000248371 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.101 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.113210+0000", 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001223607 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.215036+0000", 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101826497 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.419214+0000", 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.204177839 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.102 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:12.419944+0000", 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00072976499999999995 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:900 3.as0 3:527650bd:::smithi17829308-27:head [read 0~2686976,checksum in=4b,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:13.516193+0000", 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 17.475756620999999, 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.669168043, 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.103 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 900 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516193+0000", 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516193+0000", 2024-04-26T04:33:31.104 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516194+0000", 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.68e-06 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516200+0000", 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9070000000000004e-06 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.105 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516213+0000", 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2991e-05 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516219+0000", 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8100000000000003e-06 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516352+0000", 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000132638 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.106 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for missing object", 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.517825+0000", 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0014731670000000001 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.976957+0000", 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4591321150000001 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.977224+0000", 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026674700000000001 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.107 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:15.185361+0000", 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.208136988 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:901 3.as0 3:527650bd:::smithi17829308-27:head [read 0~2686976,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:13.516231+0000", 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 17.475718347000001, 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.673762588, 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.108 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 901 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516231+0000", 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516231+0000", 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516231+0000", 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7599999999999998e-07 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516234+0000", 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4310000000000001e-06 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516238+0000", 2024-04-26T04:33:31.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.4660000000000001e-06 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516241+0000", 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0769999999999999e-06 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.517899+0000", 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016572609999999999 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for missing object", 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.518098+0000", 2024-04-26T04:33:31.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000199271 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.977969+0000", 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.459871138 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.978307+0000", 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00033839699999999999 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:15.189994+0000", 2024-04-26T04:33:31.112 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21168617100000001 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:902 3.as0 3:527650bd:::smithi17829308-27:head [read 0~2686976,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:13.516342+0000", 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 17.475607648, 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.678722915, 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.113 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.114 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 902 2024-04-26T04:33:31.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.114 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.114 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.114 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516342+0000", 2024-04-26T04:33:31.114 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.114 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.114 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516342+0000", 2024-04-26T04:33:31.114 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516343+0000", 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.37e-06 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516352+0000", 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1430000000000004e-06 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516376+0000", 2024-04-26T04:33:31.115 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3572999999999999e-05 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.516390+0000", 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4259e-05 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.518152+0000", 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001761987 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for missing object", 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:13.518329+0000", 2024-04-26T04:33:31.116 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000176849 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.979407+0000", 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.461077577 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:14.979714+0000", 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00030723300000000002 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.117 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:15.195065+0000", 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.215350924 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:1001 3.as0 3:527650bd:::smithi17829308-27:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e89)", 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:24.508561+0000", 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 6.4833880480000001, 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.470224368, 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.118 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 1001 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:24.508561+0000", 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:24.508561+0000", 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:24.508563+0000", 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.593e-06 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:24.508571+0000", 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.5089999999999993e-06 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:24.508594+0000", 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2864e-05 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:24.508610+0000", 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5896000000000002e-05 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:24.513731+0000", 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.005120571 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:24.513857+0000", 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000126635 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:24.815393+0000", 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30153566100000001 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:24.815659+0000", 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026610199999999999 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:24.978786+0000", 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.16312653699999999 2024-04-26T04:33:31.122 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.123 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.123 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.123 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.123 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.123 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:31.124 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-26T04:33:31.252 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:31.253 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:31.253 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 0 2024-04-26T04:33:31.253 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:31.267 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-26T04:33:31.385 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:31.385 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:31.385 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:33:31.385 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:33:31.385 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:31.399 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-26T04:33:31.420 INFO:teuthology.orchestra.run.smithi012.stderr:marked in osd.10. 2024-04-26T04:33:31.435 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-26T04:33:31.447 INFO:tasks.thrashosds.thrasher:Added osd 10 2024-04-26T04:33:31.447 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-04-26T04:33:31.522 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:31.522 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:33:31.522 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:33:31.522 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:31.522 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.522 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:381 3.ds0 3:b7d7af40:::smithi17829308-48:5 [read 0~802816,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.431434+0000", 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 52.087567075999999, 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40271601499999998, 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 381 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.523 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431434+0000", 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431434+0000", 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431436+0000", 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3140000000000002e-06 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.524 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431447+0000", 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0655e-05 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431471+0000", 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4470000000000001e-05 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431488+0000", 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6129000000000001e-05 2024-04-26T04:33:31.525 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.526 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.526 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.526 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.494163+0000", 2024-04-26T04:33:31.526 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.062675393999999995 2024-04-26T04:33:31.526 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.526 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.526 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:31.526 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.494174+0000", 2024-04-26T04:33:31.526 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.114e-05 2024-04-26T04:33:31.526 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.526 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.526 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.526 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.497530+0000", 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0033555439999999998 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.497684+0000", 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015432600000000001 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.834150+0000", 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.33646604299999999 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.527 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.528 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.528 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.528 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:382 3.ds0 3:b7d7af40:::smithi17829308-48:5 [read 0~802816,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:33:31.528 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:33:31.528 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 52.087488467999997, 2024-04-26T04:33:31.528 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50469882499999996, 2024-04-26T04:33:31.528 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.528 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.528 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.528 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.528 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.528 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.528 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 382 2024-04-26T04:33:31.529 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.529 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.529 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.529 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.529 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:33:31.529 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.529 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.529 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.529 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.529 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:33:31.529 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.529 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.529 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431513+0000", 2024-04-26T04:33:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9800000000000003e-07 2024-04-26T04:33:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431521+0000", 2024-04-26T04:33:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7449999999999995e-06 2024-04-26T04:33:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.431529+0000", 2024-04-26T04:33:31.530 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.3599999999999996e-06 2024-04-26T04:33:31.531 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.531 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.531 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.531 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531644+0000", 2024-04-26T04:33:31.531 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100114514 2024-04-26T04:33:31.531 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.531 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.531 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.531 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.699490+0000", 2024-04-26T04:33:31.531 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.16784555000000001 2024-04-26T04:33:31.531 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.531 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.699653+0000", 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016372199999999999 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.936211+0000", 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23655823600000001 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.532 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:383 3.ds0 3:b7d7af40:::smithi17829308-48:5 [read 0~802816,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:39.531687+0000", 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 51.987313935000003, 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.405754104, 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 383 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.533 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531687+0000", 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531687+0000", 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531689+0000", 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.063e-06 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.534 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531702+0000", 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3156000000000001e-05 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531733+0000", 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0335999999999999e-05 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.531766+0000", 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3623e-05 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.535 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.536 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.700194+0000", 2024-04-26T04:33:31.536 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.16842744400000001 2024-04-26T04:33:31.536 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.536 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.536 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.536 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.700343+0000", 2024-04-26T04:33:31.536 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000148763 2024-04-26T04:33:31.536 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.536 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.536 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.536 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:39.937441+0000", 2024-04-26T04:33:31.536 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23709871900000001 2024-04-26T04:33:31.536 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:424 3.ds0 3:b7d7af40:::smithi17829308-48:head [append 0~720896] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.921237+0000", 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 46.597764484999999, 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40632559499999998, 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.537 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 424 2024-04-26T04:33:31.538 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.538 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.538 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.538 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.538 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.921237+0000", 2024-04-26T04:33:31.538 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.538 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.538 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.538 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.538 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.921237+0000", 2024-04-26T04:33:31.538 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.538 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.538 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.539 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.539 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.921239+0000", 2024-04-26T04:33:31.539 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2900000000000001e-06 2024-04-26T04:33:31.539 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.539 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.539 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.539 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922033+0000", 2024-04-26T04:33:31.539 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00079434500000000001 2024-04-26T04:33:31.539 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.539 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.539 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.539 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922059+0000", 2024-04-26T04:33:31.540 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5925999999999999e-05 2024-04-26T04:33:31.540 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.540 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.540 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.540 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922072+0000", 2024-04-26T04:33:31.540 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2943e-05 2024-04-26T04:33:31.540 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.540 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.540 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.540 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922191+0000", 2024-04-26T04:33:31.540 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000118581 2024-04-26T04:33:31.540 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.540 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922362+0000", 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017091599999999999 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.924002+0000", 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016405359999999999 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.928906+0000", 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0049035720000000001 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.541 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.542 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.542 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327373+0000", 2024-04-26T04:33:31.542 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39846698000000003 2024-04-26T04:33:31.542 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.542 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.542 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.542 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.327562+0000", 2024-04-26T04:33:31.542 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018950600000000001 2024-04-26T04:33:31.542 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.542 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.542 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.542 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.542 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:425 3.ds0 3:b7d7af40:::smithi17829308-48:head [append 0~49152] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.922098+0000", 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 46.596902641, 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.406108466, 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 425 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.543 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922098+0000", 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922098+0000", 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922100+0000", 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2300000000000001e-06 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.544 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.545 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.545 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922189+0000", 2024-04-26T04:33:31.545 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9623999999999994e-05 2024-04-26T04:33:31.545 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.545 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.545 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.545 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922208+0000", 2024-04-26T04:33:31.545 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8643000000000001e-05 2024-04-26T04:33:31.545 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.545 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.545 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.545 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922219+0000", 2024-04-26T04:33:31.545 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0699999999999999e-05 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.925187+0000", 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0029683930000000002 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.925371+0000", 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000184382 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.546 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.926426+0000", 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0010545190000000001 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.928909+0000", 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.002482981 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328000+0000", 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.399091539 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.547 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328207+0000", 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000206455 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:426 3.ds0 3:b7d7af40:::smithi17829308-48:head [setxattr _header (56) in=7b] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.922241+0000", 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 46.596760484999997, 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40635279400000002, 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.548 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 426 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922241+0000", 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922241+0000", 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.549 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.550 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.550 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.550 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922242+0000", 2024-04-26T04:33:31.550 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0470000000000001e-06 2024-04-26T04:33:31.550 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.550 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.550 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.550 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922246+0000", 2024-04-26T04:33:31.550 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5029999999999999e-06 2024-04-26T04:33:31.550 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.550 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.550 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.550 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922257+0000", 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.059e-05 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922265+0000", 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.8869999999999992e-06 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.927201+0000", 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0049363640000000004 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.551 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.927353+0000", 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015164000000000001 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.928331+0000", 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00097882500000000001 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.029157+0000", 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100825282 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.552 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328408+0000", 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29925099700000002 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.328593+0000", 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018565899999999999 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:427 3.ds0 3:b7d7af40:::smithi17829308-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e50)", 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:44.922276+0000", 2024-04-26T04:33:31.553 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 46.596725378000002, 2024-04-26T04:33:31.554 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.57447843799999998, 2024-04-26T04:33:31.554 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.554 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.554 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.554 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.554 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.554 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.554 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 427 2024-04-26T04:33:31.554 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.554 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.554 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.554 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.554 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922276+0000", 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922276+0000", 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922277+0000", 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.7000000000000003e-07 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.555 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922280+0000", 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.0589999999999998e-06 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922293+0000", 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3617e-05 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.922302+0000", 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4300000000000006e-06 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.556 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.929362+0000", 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.007060439 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:44.929534+0000", 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017232900000000001 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.329547+0000", 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40001209799999998 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.557 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.329786+0000", 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00023956800000000001 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:45.496754+0000", 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.16696802799999999 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.558 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 56/30 ECSubWrite(tid=60, reqid=client.4670.0:478, at_version=56'42, trim_to=0'0, roll_forward_to=50'39))", 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.321773+0000", 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 41.197228516999999, 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50217973100000002, 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321773+0000", 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.559 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321773+0000", 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321773+0000", 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.4399999999999999e-07 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321891+0000", 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011750499999999999 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.560 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321903+0000", 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.255e-05 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.421985+0000", 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100081395 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.522172+0000", 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10018666 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.561 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.522181+0000", 2024-04-26T04:33:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8349999999999999e-06 2024-04-26T04:33:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823898+0000", 2024-04-26T04:33:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30171669200000001 2024-04-26T04:33:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823952+0000", 2024-04-26T04:33:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4249999999999997e-05 2024-04-26T04:33:31.562 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 56/30 ECSubWrite(tid=61, reqid=client.4670.0:479, at_version=56'43, trim_to=0'0, roll_forward_to=50'39))", 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.422026+0000", 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 41.096975243999999, 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40196675300000001, 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.563 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.564 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422026+0000", 2024-04-26T04:33:31.564 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.564 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.564 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.564 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422026+0000", 2024-04-26T04:33:31.564 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.564 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.564 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.564 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422026+0000", 2024-04-26T04:33:31.564 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.6599999999999996e-07 2024-04-26T04:33:31.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.565 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.565 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.565 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422072+0000", 2024-04-26T04:33:31.565 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5596000000000002e-05 2024-04-26T04:33:31.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.565 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.565 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.565 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422086+0000", 2024-04-26T04:33:31.565 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3577000000000001e-05 2024-04-26T04:33:31.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.565 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.565 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.565 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422094+0000", 2024-04-26T04:33:31.566 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.6319999999999997e-06 2024-04-26T04:33:31.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.566 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.566 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.622832+0000", 2024-04-26T04:33:31.566 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20073786099999999 2024-04-26T04:33:31.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.566 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.566 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.622864+0000", 2024-04-26T04:33:31.566 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1495000000000003e-05 2024-04-26T04:33:31.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823965+0000", 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20110186299999999 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823993+0000", 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7163000000000001e-05 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.567 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWrite(3.5s4 56/30 ECSubWrite(tid=62, reqid=client.4670.0:480, at_version=56'44, trim_to=0'0, roll_forward_to=50'39))", 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.422105+0000", 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 41.096895699999997, 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40191106599999998, 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422105+0000", 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.568 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.569 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422105+0000", 2024-04-26T04:33:31.569 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.569 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.569 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.569 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422106+0000", 2024-04-26T04:33:31.569 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1399999999999998e-07 2024-04-26T04:33:31.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.569 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.569 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.569 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422115+0000", 2024-04-26T04:33:31.569 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.5319999999999995e-06 2024-04-26T04:33:31.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.570 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.570 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422130+0000", 2024-04-26T04:33:31.570 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4474999999999999e-05 2024-04-26T04:33:31.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.570 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.570 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.422136+0000", 2024-04-26T04:33:31.570 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7039999999999997e-06 2024-04-26T04:33:31.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.570 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.570 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.623647+0000", 2024-04-26T04:33:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201510879 2024-04-26T04:33:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.623662+0000", 2024-04-26T04:33:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5122e-05 2024-04-26T04:33:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.823997+0000", 2024-04-26T04:33:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.200334135 2024-04-26T04:33:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.824016+0000", 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9905e-05 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:635 3.ds0 3:b761f772:::smithi17829308-43:head [create,setxattr ]PaibOQQZPXlPQ_TVRVsq`]`lQnrU_adXdhR`ia_ (33) in=40b,setxattr mga[\\pjjlr_f^pjpoNm[SiQinb`p]Uljsr\\PZamg (1614) in=40b,setxattr \\\\nNdqV]P_aoSQ]T][SZi`\\`emqRalsThkWkp]_O (1998) in=40b,setxattr QPRVcO^PYksRnjf^RqtasbqiemoSYpeimc]_kQOo (1505) in=40b,setxattr i`dWfsdYVZUqQmWleQeniob`plYiVWUWSsnsd`\\a (1017) in=40b,setxattr Qn`WN^VrYNfSm_nWkOlO[bZ_[Zci__kgsaZeg[_R (1614) in=40b,setxattr \\]NdaY]dOTpk\\bgtmgifQtgWtaqfd]r`TbktPRjh (19016) in=40b,setxattr NlQZaoabhi (349) in=10b,setxattr QbZ^bqgnd]QnP\\ZjeRjaP[UfZgdm[V^sRaRpnZYY (1963) in=40b,setxattr `]YWO_jgNfUs_SQ`RaqUhXmNY[TtZZbaUj\\NPtY^ (1190) in=40b,setxattr dnfqsNhbe]^ml]ekS[ST`fnk_kpiTUa]Y]ges`rW (17824) in=40b,setxattr RgOUnWhYeYcgnikgiQQWNr_OWg_k`YSaN^XQOdPc (1437) in=40b,setxattr QUmNqZgRZOU]WaWWgl\\pnRhtilqS`cQaScmgUp_e (1438) in=40b,setxattr TfW^Smr_geRfti]efnPdr`iWQUikeXOp`StYqtQd (12280) in=40b,setxattr ZtZ (349) in=3b,setxattr NQYfjd\\ (349) in=7b,setxattr aNaTblXeObdSN[ahZn`j[mcbUaSjeXOnc[dZ^ZZo (910) in=40b,setxattr deb]iS_OUlbmWR_sP^er[TdaV`cSqnnthSotq]YZ (7928) in=40b,setxattr ^s_gVdo]kXUSXb]aqYjbUYbRmcoYksXSWQdVsh (349) in=38b,setxattr h^YTbW\\dli\\^dffQaXRNaZQVkZPONqhhabmhR_[s (1822) in=40b] snapc e=[e,b] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:56.110252+0000", 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 35.408748801999998, 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40734936900000002, 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.572 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 635 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110252+0000", 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.573 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110252+0000", 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110255+0000", 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6560000000000001e-06 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110399+0000", 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000144098 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110437+0000", 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7867000000000002e-05 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110447+0000", 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0183000000000001e-05 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110619+0000", 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017148900000000001 2024-04-26T04:33:31.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.110847+0000", 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000228199 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.112826+0000", 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0019796530000000001 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.115968+0000", 2024-04-26T04:33:31.576 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.003141657 2024-04-26T04:33:31.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.577 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.577 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.517378+0000", 2024-04-26T04:33:31.577 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.401409617 2024-04-26T04:33:31.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.577 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.577 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:56.517602+0000", 2024-04-26T04:33:31.577 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022395 2024-04-26T04:33:31.577 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.577 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.577 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.578 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:740 3.ds0 3:bd5491ce:::smithi17829308-20:head [append 0~507904] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:31.578 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.364648+0000", 2024-04-26T04:33:31.578 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 29.1543533, 2024-04-26T04:33:31.578 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.72537579699999999, 2024-04-26T04:33:31.578 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.578 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.578 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.578 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.578 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.578 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.578 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 740 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.364648+0000", 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.364648+0000", 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.579 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.580 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.364652+0000", 2024-04-26T04:33:31.580 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.7100000000000001e-06 2024-04-26T04:33:31.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.580 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.580 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.364987+0000", 2024-04-26T04:33:31.580 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00033577699999999999 2024-04-26T04:33:31.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.580 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.580 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.365026+0000", 2024-04-26T04:33:31.580 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.8359999999999999e-05 2024-04-26T04:33:31.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.365044+0000", 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8578999999999999e-05 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.365197+0000", 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015245800000000001 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.365211+0000", 2024-04-26T04:33:31.581 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4303e-05 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.468736+0000", 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10352488 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.468920+0000", 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018384299999999999 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.582 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.471244+0000", 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0023246579999999998 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.477358+0000", 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0061135470000000004 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.089886+0000", 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61252856499999997 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.583 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.090024+0000", 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000137117 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:741 3.ds0 3:bd5491ce:::smithi17829308-20:head [append 0~425984] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.368769+0000", 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 29.150231951999999, 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.72154272200000003, 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.584 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 741 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.368769+0000", 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.368769+0000", 2024-04-26T04:33:31.586 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.586 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.586 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.368772+0000", 2024-04-26T04:33:31.586 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.503e-06 2024-04-26T04:33:31.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.586 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.586 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.369295+0000", 2024-04-26T04:33:31.586 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00052320399999999999 2024-04-26T04:33:31.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.369324+0000", 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9260000000000001e-05 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.369339+0000", 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5079000000000001e-05 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.472521+0000", 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10318134900000001 2024-04-26T04:33:31.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.472684+0000", 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016300099999999999 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.474045+0000", 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0013610339999999999 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.480049+0000", 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0060043520000000001 2024-04-26T04:33:31.588 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.090206+0000", 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61015733800000005 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.090312+0000", 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000105602 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.589 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:742 3.ds0 3:bd5491ce:::smithi17829308-20:head [append 0~442368] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:31.590 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.372518+0000", 2024-04-26T04:33:31.590 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 29.146483009000001, 2024-04-26T04:33:31.590 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81824782500000004, 2024-04-26T04:33:31.590 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.590 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.590 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.590 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.590 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.590 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.590 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 742 2024-04-26T04:33:31.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.590 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.372518+0000", 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.372518+0000", 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.372520+0000", 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1440000000000001e-06 2024-04-26T04:33:31.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.373038+0000", 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00051735099999999999 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.373070+0000", 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2215999999999998e-05 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.373086+0000", 2024-04-26T04:33:31.592 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5817000000000001e-05 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.474948+0000", 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101862724 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.475105+0000", 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015690300000000001 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.593 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.476472+0000", 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0013667810000000001 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.480143+0000", 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0036706120000000002 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.190647+0000", 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71050477899999998 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.594 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.190766+0000", 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000118498 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:743 3.ds0 3:bd5491ce:::smithi17829308-20:head [append 0~540672] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.376986+0000", 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 29.14201542, 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81437894700000002, 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.595 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.596 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.596 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.596 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.596 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 743 2024-04-26T04:33:31.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.596 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.596 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.596 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.376986+0000", 2024-04-26T04:33:31.596 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.596 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.376986+0000", 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.376988+0000", 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2400000000000002e-06 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.377588+0000", 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00059970599999999998 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.598 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.598 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.377621+0000", 2024-04-26T04:33:31.598 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3402999999999999e-05 2024-04-26T04:33:31.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.598 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.598 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.377637+0000", 2024-04-26T04:33:31.598 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5489999999999999e-05 2024-04-26T04:33:31.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.598 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.598 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.477443+0000", 2024-04-26T04:33:31.598 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.099806487999999999 2024-04-26T04:33:31.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.599 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.599 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.477613+0000", 2024-04-26T04:33:31.599 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017017399999999999 2024-04-26T04:33:31.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.599 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.599 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.479104+0000", 2024-04-26T04:33:31.599 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0014903480000000001 2024-04-26T04:33:31.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.599 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.684294+0000", 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20519051999999999 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.191246+0000", 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50695185700000001 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.191365+0000", 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000118721 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.600 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:744 3.ds0 3:bd5491ce:::smithi17829308-20:head [append 0~147456] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.378092+0000", 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 29.140909162, 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.813543248, 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.601 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 744 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378092+0000", 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378092+0000", 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.602 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378095+0000", 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.667e-06 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378258+0000", 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016352399999999999 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378291+0000", 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.2601999999999997e-05 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378306+0000", 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5307000000000001e-05 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.580438+0000", 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20213155699999999 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.580618+0000", 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018046699999999999 2024-04-26T04:33:31.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.581709+0000", 2024-04-26T04:33:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001090466 2024-04-26T04:33:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.684369+0000", 2024-04-26T04:33:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.102660494 2024-04-26T04:33:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.605 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.191539+0000", 2024-04-26T04:33:31.606 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.507170122 2024-04-26T04:33:31.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.606 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.606 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.191635+0000", 2024-04-26T04:33:31.606 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6042e-05 2024-04-26T04:33:31.606 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.606 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.606 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.606 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.606 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:745 3.ds0 3:bd5491ce:::smithi17829308-20:head [setxattr _header (57) in=7b] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:31.606 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.378334+0000", 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 29.140667376, 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81367699000000004, 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 745 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.607 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378334+0000", 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378334+0000", 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378335+0000", 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.257e-06 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.608 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378343+0000", 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5789999999999998e-06 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378358+0000", 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4987000000000001e-05 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378369+0000", 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1353e-05 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.609 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.582538+0000", 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20416878699999999 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.582700+0000", 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016191 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.583551+0000", 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00085183999999999998 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.610 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.684372+0000", 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100820914 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.191891+0000", 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.507518468 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.192011+0000", 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000119895 2024-04-26T04:33:31.611 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:746 3.ds0 3:bd5491ce:::smithi17829308-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.378386+0000", 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 29.140615597, 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2193996330000001, 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:31.612 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 746 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378386+0000", 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378386+0000", 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.613 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378386+0000", 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.1699999999999997e-07 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378390+0000", 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.4400000000000001e-06 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378403+0000", 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3686000000000001e-05 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.378414+0000", 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0976e-05 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.192805+0000", 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81439069900000005 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.193205+0000", 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00040003400000000001 2024-04-26T04:33:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.597785+0000", 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40457998099999998 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 67/30 ECSubWriteReply(tid=64, last_complete=67'42, committed=1, applied=1))", 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.478796+0000", 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 29.040205209, 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30610332899999998, 2024-04-26T04:33:31.616 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.478796+0000", 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.478796+0000", 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.618 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.618 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.478797+0000", 2024-04-26T04:33:31.618 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.573e-06 2024-04-26T04:33:31.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.618 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.618 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.478808+0000", 2024-04-26T04:33:31.618 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.096e-05 2024-04-26T04:33:31.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.618 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.618 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.478828+0000", 2024-04-26T04:33:31.618 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9964000000000001e-05 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.578921+0000", 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10009237 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.784875+0000", 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.205954516 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.619 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.784899+0000", 2024-04-26T04:33:31.620 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3946000000000001e-05 2024-04-26T04:33:31.620 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.620 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.620 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.620 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.620 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "MOSDECSubOpRead(3.ds0 67/30 ECSubRead(tid=70, to_read={3:bd5491ce:::smithi17829308-20:head=0,4096,0}, subchunks={3:bd5491ce:::smithi17829308-20:head=[0,1]}, attrs_to_read=))", 2024-04-26T04:33:31.620 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:03.193748+0000", 2024-04-26T04:33:31.620 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 28.325252677000002, 2024-04-26T04:33:31.620 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30233241900000002, 2024-04-26T04:33:31.620 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:31.620 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:31.620 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "reached pg", 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.193748+0000", 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.193748+0000", 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:31.621 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.193748+0000", 2024-04-26T04:33:31.622 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.622 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:31.622 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.193748+0000", 2024-04-26T04:33:31.622 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:31.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.622 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:31.622 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.193790+0000", 2024-04-26T04:33:31.622 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.1279000000000002e-05 2024-04-26T04:33:31.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.623 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:31.623 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.293934+0000", 2024-04-26T04:33:31.623 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100144097 2024-04-26T04:33:31.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.623 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:31.623 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.495906+0000", 2024-04-26T04:33:31.623 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201972179 2024-04-26T04:33:31.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:31.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:31.623 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:31.623 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.496081+0000", 2024-04-26T04:33:31.624 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000174864 2024-04-26T04:33:31.624 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.624 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.624 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.624 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:31.624 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:31.624 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:31.625 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-26T04:33:31.630 INFO:teuthology.orchestra.run.smithi012.stdout:mon.a: {} 2024-04-26T04:33:31.631 INFO:teuthology.orchestra.run.smithi012.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-26T04:33:31.643 INFO:teuthology.orchestra.run.smithi012.stdout:mon.b: {} 2024-04-26T04:33:31.643 INFO:teuthology.orchestra.run.smithi012.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-26T04:33:31.657 INFO:teuthology.orchestra.run.smithi012.stdout:mon.c: {} 2024-04-26T04:33:31.658 INFO:teuthology.orchestra.run.smithi012.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-26T04:33:31.675 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-26T04:33:31.753 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:31.753 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:31.754 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 0 2024-04-26T04:33:31.754 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:31.768 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-26T04:33:31.886 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:31.886 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:31.886 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:33:31.886 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:33:31.886 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:31.902 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-26T04:33:32.030 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:32.030 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:33:32.030 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:33:32.030 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:32.030 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.030 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:8 3.fs0 3:f50d0def:::smithi17829308-3:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:32.030 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:33:32.030 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 66.825782124, 2024-04-26T04:33:32.030 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51428121599999999, 2024-04-26T04:33:32.030 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.030 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 8 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.031 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200418+0000", 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200419+0000", 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9800000000000002e-07 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200433+0000", 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4513e-05 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200460+0000", 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6438999999999999e-05 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200477+0000", 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7346e-05 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200757+0000", 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000279771 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.033 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.034 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200766+0000", 2024-04-26T04:33:32.034 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.7159999999999992e-06 2024-04-26T04:33:32.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.034 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.034 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.034 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.341113+0000", 2024-04-26T04:33:32.034 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14034733799999999 2024-04-26T04:33:32.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.034 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.034 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.034 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.341282+0000", 2024-04-26T04:33:32.034 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000169197 2024-04-26T04:33:32.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.342226+0000", 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00094361300000000005 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.343875+0000", 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0016493910000000001 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714472+0000", 2024-04-26T04:33:32.035 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.37059684599999998 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714699+0000", 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022704800000000001 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:9 3.fs0 3:f50d0def:::smithi17829308-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 66.825701434999999, 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71712659899999998, 2024-04-26T04:33:32.036 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.037 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.037 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:32.037 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.037 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.037 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.037 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 9 2024-04-26T04:33:32.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.037 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.037 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.037 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:33:32.037 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200499+0000", 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1999999999999998e-07 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200506+0000", 2024-04-26T04:33:32.038 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.1399999999999997e-06 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200520+0000", 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4253000000000001e-05 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200531+0000", 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1375999999999999e-05 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.039 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200817+0000", 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00028585699999999998 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.200824+0000", 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6730000000000003e-06 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.342845+0000", 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.142020961 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:32.040 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.343080+0000", 2024-04-26T04:33:32.041 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000235481 2024-04-26T04:33:32.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.041 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.041 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.714851+0000", 2024-04-26T04:33:32.041 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.371770347 2024-04-26T04:33:32.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.041 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.041 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.715101+0000", 2024-04-26T04:33:32.041 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00025083100000000002 2024-04-26T04:33:32.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.042 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.042 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917625+0000", 2024-04-26T04:33:32.042 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20252376 2024-04-26T04:33:32.042 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.042 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.042 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.042 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.042 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:16 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~98304,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:32.042 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:33:32.042 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 66.824544434000003, 2024-04-26T04:33:32.042 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.514661445, 2024-04-26T04:33:32.042 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 16 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.043 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.044 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.044 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201656+0000", 2024-04-26T04:33:32.044 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.044 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.044 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201657+0000", 2024-04-26T04:33:32.044 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.052e-06 2024-04-26T04:33:32.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.044 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.044 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201822+0000", 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016550000000000001 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201842+0000", 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0231e-05 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201856+0000", 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3234999999999999e-05 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.045 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201960+0000", 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0001043 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201982+0000", 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1936000000000001e-05 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344166+0000", 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14218376399999999 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.344545+0000", 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00037956399999999999 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.346210+0000", 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00166501 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349256+0000", 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0030461080000000001 2024-04-26T04:33:32.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716126+0000", 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.366869679 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716317+0000", 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019106599999999999 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:17 3.fs0 3:f03878c8:::smithi17829308-6:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 66.824316343000007, 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51482183299999995, 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 17 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.049 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201884+0000", 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201885+0000", 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1060000000000001e-06 2024-04-26T04:33:32.050 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.051 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.051 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201891+0000", 2024-04-26T04:33:32.051 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.806e-06 2024-04-26T04:33:32.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.051 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.051 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201905+0000", 2024-04-26T04:33:32.051 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.432e-05 2024-04-26T04:33:32.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.051 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201929+0000", 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4417e-05 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202056+0000", 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012684400000000001 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202066+0000", 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.5230000000000001e-06 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.052 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.053 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.053 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.347195+0000", 2024-04-26T04:33:32.053 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.145129646 2024-04-26T04:33:32.053 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.053 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.053 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.053 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.347408+0000", 2024-04-26T04:33:32.053 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00021322299999999999 2024-04-26T04:33:32.053 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.053 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.053 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.053 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.348249+0000", 2024-04-26T04:33:32.053 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000840363 2024-04-26T04:33:32.054 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.054 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.054 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.054 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.353641+0000", 2024-04-26T04:33:32.054 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0053921280000000004 2024-04-26T04:33:32.054 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.054 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.054 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.054 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716520+0000", 2024-04-26T04:33:32.054 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.36287924799999999 2024-04-26T04:33:32.054 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716705+0000", 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018520900000000001 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:18 3.fs0 3:f03878c8:::smithi17829308-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 66.824253214999999, 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71589294699999995, 2024-04-26T04:33:32.055 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.056 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.056 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:32.056 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.056 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.056 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.056 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 18 2024-04-26T04:33:32.056 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.056 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.056 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.056 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.056 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:33:32.056 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.056 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201947+0000", 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201948+0000", 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.11e-06 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201962+0000", 2024-04-26T04:33:32.057 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4297999999999999e-05 2024-04-26T04:33:32.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.058 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.058 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.058 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.201990+0000", 2024-04-26T04:33:32.058 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8243e-05 2024-04-26T04:33:32.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.058 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.058 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.058 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202004+0000", 2024-04-26T04:33:32.058 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3259e-05 2024-04-26T04:33:32.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.058 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.058 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202132+0000", 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000128596 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.202149+0000", 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6463000000000001e-05 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349005+0000", 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.14685641099999999 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.060 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:32.060 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.349233+0000", 2024-04-26T04:33:32.060 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000227798 2024-04-26T04:33:32.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.060 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.060 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.060 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716805+0000", 2024-04-26T04:33:32.060 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.36757169099999998 2024-04-26T04:33:32.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.060 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.060 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.060 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.716944+0000", 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013965200000000001 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.917840+0000", 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20089542599999999 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:346 3.fs0 3:fa2b8868:::smithi17829308-13:head [copy-from ver 14] snapc 4=[2] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.181230+0000", 2024-04-26T04:33:32.061 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 56.844970435, 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.80960196600000001, 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 346 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181230+0000", 2024-04-26T04:33:32.062 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.063 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.063 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181230+0000", 2024-04-26T04:33:32.063 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.063 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.063 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181232+0000", 2024-04-26T04:33:32.063 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3609999999999999e-06 2024-04-26T04:33:32.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181257+0000", 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5100999999999999e-05 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181283+0000", 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.5612000000000001e-05 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181342+0000", 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.9744000000000003e-05 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.065 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.065 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248343+0000", 2024-04-26T04:33:32.065 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.067000834999999995 2024-04-26T04:33:32.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.065 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.065 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.065 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248487+0000", 2024-04-26T04:33:32.065 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014416900000000001 2024-04-26T04:33:32.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.065 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.065 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.066 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.787056+0000", 2024-04-26T04:33:32.066 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.53856828300000004 2024-04-26T04:33:32.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.066 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.066 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.066 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.789372+0000", 2024-04-26T04:33:32.066 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0023167069999999999 2024-04-26T04:33:32.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.066 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.066 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.066 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.990698+0000", 2024-04-26T04:33:32.066 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201325688 2024-04-26T04:33:32.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.067 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.067 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.067 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.990831+0000", 2024-04-26T04:33:32.067 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000133466 2024-04-26T04:33:32.067 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.067 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.067 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.067 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.067 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.067 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:347 3.fs0 3:fa2b8868:::smithi17829308-13:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e40)", 2024-04-26T04:33:32.067 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.181534+0000", 2024-04-26T04:33:32.067 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 56.844665876999997, 2024-04-26T04:33:32.067 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.60738179000000003, 2024-04-26T04:33:32.068 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.068 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.068 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:32.068 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.068 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.068 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.068 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 347 2024-04-26T04:33:32.068 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.068 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.068 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.068 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.068 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181534+0000", 2024-04-26T04:33:32.069 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.069 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.069 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.069 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181534+0000", 2024-04-26T04:33:32.069 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.069 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.069 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.069 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181535+0000", 2024-04-26T04:33:32.069 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0640000000000001e-06 2024-04-26T04:33:32.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181541+0000", 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4099999999999999e-06 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181557+0000", 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6728999999999999e-05 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.181570+0000", 2024-04-26T04:33:32.070 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2743e-05 2024-04-26T04:33:32.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.071 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.071 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248864+0000", 2024-04-26T04:33:32.071 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.067293511 2024-04-26T04:33:32.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.071 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:32.071 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.248995+0000", 2024-04-26T04:33:32.071 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013142000000000001 2024-04-26T04:33:32.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.071 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.788613+0000", 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.53961845799999997 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.788760+0000", 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014664600000000001 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.788916+0000", 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000155809 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.072 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:430 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~622592] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.182041+0000", 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 45.844158907999997, 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51141810899999995, 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.073 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 430 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182041+0000", 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182041+0000", 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.074 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182044+0000", 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.83e-06 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182758+0000", 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00071432400000000001 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.182792+0000", 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3655999999999998e-05 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.282928+0000", 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100136005 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.283058+0000", 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013049700000000001 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.283071+0000", 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3106e-05 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.077 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.077 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.326367+0000", 2024-04-26T04:33:32.077 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.043295412999999998 2024-04-26T04:33:32.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.077 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.077 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.326694+0000", 2024-04-26T04:33:32.077 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00032741600000000002 2024-04-26T04:33:32.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.077 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.077 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.329727+0000", 2024-04-26T04:33:32.077 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0030326509999999999 2024-04-26T04:33:32.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.078 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.078 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.333822+0000", 2024-04-26T04:33:32.078 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0040948139999999996 2024-04-26T04:33:32.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.078 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.078 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.693241+0000", 2024-04-26T04:33:32.078 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.35941913199999997 2024-04-26T04:33:32.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.693459+0000", 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00021826500000000001 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:434 3.fs0 3:f03878c8:::smithi17829308-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e52)", 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:46.384120+0000", 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 45.642080059000001, 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81328199499999998, 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.079 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.080 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:32.080 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.080 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.080 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.080 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 434 2024-04-26T04:33:32.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.080 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.080 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.080 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384120+0000", 2024-04-26T04:33:32.080 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.080 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384120+0000", 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384121+0000", 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3069999999999999e-06 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384127+0000", 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4430000000000002e-06 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.082 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.082 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384147+0000", 2024-04-26T04:33:32.082 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0533000000000001e-05 2024-04-26T04:33:32.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.082 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.082 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.384163+0000", 2024-04-26T04:33:32.082 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5948999999999999e-05 2024-04-26T04:33:32.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.082 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.082 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.491986+0000", 2024-04-26T04:33:32.082 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.107822418 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.492246+0000", 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026025100000000003 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.796841+0000", 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30459499299999998 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.083 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:46.796996+0000", 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015495 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:47.197402+0000", 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40040615099999999 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:488 3.fs0 3:f8001b32:::smithi17829308-38:head [copy-from ver 18] snapc d=[b,a] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:51.211447+0000", 2024-04-26T04:33:32.084 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 40.814753404999998, 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71306741799999995, 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 488 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211447+0000", 2024-04-26T04:33:32.085 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211447+0000", 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211449+0000", 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.554e-06 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.086 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211474+0000", 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4987999999999999e-05 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211499+0000", 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4847000000000001e-05 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211516+0000", 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6731e-05 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.087 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211811+0000", 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000294999 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211828+0000", 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7842999999999998e-05 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.415705+0000", 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20387643899999999 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.088 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.416026+0000", 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00032070999999999998 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.819320+0000", 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40329431799999998 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.821274+0000", 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001954258 2024-04-26T04:33:32.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.924373+0000", 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103099154 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.924514+0000", 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014057699999999999 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.090 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:489 3.fs0 3:f8001b32:::smithi17829308-38:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e57)", 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:51.211561+0000", 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 40.814639194999998, 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61242426699999997, 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 489 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.091 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211561+0000", 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211561+0000", 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211561+0000", 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1699999999999997e-07 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.092 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211566+0000", 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.2289999999999997e-06 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211576+0000", 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0543999999999999e-05 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211585+0000", 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1749999999999994e-06 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.093 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211947+0000", 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00036154399999999999 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.211960+0000", 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.3443e-05 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.615185+0000", 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40322487499999998 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.094 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for blocked object", 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.615237+0000", 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.2206999999999997e-05 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.823589+0000", 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.208351707 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.823795+0000", 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00020569300000000001 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.095 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.823985+0000", 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019013299999999999 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:501 3.fs0 3:f8001b32:::smithi17829308-38:head [copy-from ver 2] snapc d=[b] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:52.384346+0000", 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 39.641853588000004, 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.611539218, 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.096 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 501 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384346+0000", 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.097 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384346+0000", 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384349+0000", 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3760000000000002e-06 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384360+0000", 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1240000000000001e-05 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.098 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384436+0000", 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5871999999999995e-05 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.384464+0000", 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.7693000000000001e-05 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.395288+0000", 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.010824926 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.099 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.395451+0000", 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000162607 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.699338+0000", 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.303887405 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.792390+0000", 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.093051153999999997 2024-04-26T04:33:32.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.995723+0000", 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.203333075 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:52.995886+0000", 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016286999999999999 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.101 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.102 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:817 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~638976] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:32.102 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.281200+0000", 2024-04-26T04:33:32.102 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 24.745000345000001, 2024-04-26T04:33:32.102 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51066042700000003, 2024-04-26T04:33:32.102 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.102 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.102 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.102 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.102 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.102 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.102 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 817 2024-04-26T04:33:32.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.102 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.103 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.103 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.281200+0000", 2024-04-26T04:33:32.103 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.103 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.103 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.281200+0000", 2024-04-26T04:33:32.103 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.103 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.103 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.103 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.281202+0000", 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8949999999999998e-06 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.281629+0000", 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00042693800000000002 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.281661+0000", 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.1429e-05 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.104 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.381806+0000", 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100144873 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382002+0000", 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019577700000000001 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382014+0000", 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2694000000000001e-05 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.105 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.483589+0000", 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101574648 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.483736+0000", 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014714400000000001 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.485464+0000", 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001728151 2024-04-26T04:33:32.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.107 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.107 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.593922+0000", 2024-04-26T04:33:32.107 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10845802 2024-04-26T04:33:32.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.107 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.107 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.791793+0000", 2024-04-26T04:33:32.107 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.19787133000000001 2024-04-26T04:33:32.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.107 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.107 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.107 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.791860+0000", 2024-04-26T04:33:32.108 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6527999999999997e-05 2024-04-26T04:33:32.108 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.108 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.108 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.108 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.108 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:819 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~802816] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:32.108 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.382377+0000", 2024-04-26T04:33:32.108 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 24.643822848999999, 2024-04-26T04:33:32.108 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.510421554, 2024-04-26T04:33:32.108 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.108 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.108 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 819 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382377+0000", 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.109 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382377+0000", 2024-04-26T04:33:32.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.110 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.110 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382378+0000", 2024-04-26T04:33:32.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.75e-07 2024-04-26T04:33:32.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.110 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.110 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.110 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382790+0000", 2024-04-26T04:33:32.110 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00041200200000000002 2024-04-26T04:33:32.110 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382799+0000", 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6749999999999997e-06 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382804+0000", 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5109999999999997e-06 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.482975+0000", 2024-04-26T04:33:32.111 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10017138 2024-04-26T04:33:32.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.112 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.112 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.482987+0000", 2024-04-26T04:33:32.112 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.163e-05 2024-04-26T04:33:32.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.112 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.112 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.112 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.490398+0000", 2024-04-26T04:33:32.112 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0074111580000000002 2024-04-26T04:33:32.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.490664+0000", 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000265596 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.492500+0000", 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001835857 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.593962+0000", 2024-04-26T04:33:32.113 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10146279800000001 2024-04-26T04:33:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.892700+0000", 2024-04-26T04:33:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.298737366 2024-04-26T04:33:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.892799+0000", 2024-04-26T04:33:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.8906000000000005e-05 2024-04-26T04:33:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.114 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.115 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.115 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:820 3.fs0 3:f03878c8:::smithi17829308-6:head [append 0~81920] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:32.115 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.382815+0000", 2024-04-26T04:33:32.115 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 24.643384513000001, 2024-04-26T04:33:32.115 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61062931799999998, 2024-04-26T04:33:32.115 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.115 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 820 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382815+0000", 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.116 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382815+0000", 2024-04-26T04:33:32.117 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.117 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.117 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382816+0000", 2024-04-26T04:33:32.117 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8400000000000004e-07 2024-04-26T04:33:32.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.117 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.117 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382850+0000", 2024-04-26T04:33:32.117 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3955999999999998e-05 2024-04-26T04:33:32.117 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.117 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382857+0000", 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.0210000000000002e-06 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382861+0000", 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.9489999999999998e-06 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.483105+0000", 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100243689 2024-04-26T04:33:32.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.483116+0000", 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0943e-05 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.594521+0000", 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.11140509699999999 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.594688+0000", 2024-04-26T04:33:32.119 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016764500000000001 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.595346+0000", 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00065723599999999995 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.596772+0000", 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001426411 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.120 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.993251+0000", 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39647891000000002 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.993445+0000", 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00019387699999999999 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:821 3.fs0 3:f03878c8:::smithi17829308-6:head [setxattr _header (56) in=7b] snapc 14=[14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.382870+0000", 2024-04-26T04:33:32.121 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 24.643330371000001, 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61114189399999996, 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 821 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382870+0000", 2024-04-26T04:33:32.122 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.123 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.123 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382870+0000", 2024-04-26T04:33:32.123 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.123 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.123 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382870+0000", 2024-04-26T04:33:32.123 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.9099999999999999e-07 2024-04-26T04:33:32.123 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.123 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.123 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382872+0000", 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.305e-06 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382876+0000", 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.8380000000000004e-06 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382879+0000", 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8459999999999999e-06 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.124 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.125 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.595741+0000", 2024-04-26T04:33:32.125 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21286179199999999 2024-04-26T04:33:32.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.125 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.125 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.596013+0000", 2024-04-26T04:33:32.125 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00027180599999999998 2024-04-26T04:33:32.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.125 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.125 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.125 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.596574+0000", 2024-04-26T04:33:32.125 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00056180100000000001 2024-04-26T04:33:32.125 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.596964+0000", 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00038925 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.993808+0000", 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39684441399999998 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.994011+0000", 2024-04-26T04:33:32.126 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000203451 2024-04-26T04:33:32.127 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.127 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.127 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.127 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.127 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:822 3.fs0 3:f03878c8:::smithi17829308-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:32.127 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.382891+0000", 2024-04-26T04:33:32.127 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 24.643308558000001, 2024-04-26T04:33:32.127 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.81435076900000003, 2024-04-26T04:33:32.127 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.127 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.127 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:32.127 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 822 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382891+0000", 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382891+0000", 2024-04-26T04:33:32.128 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.129 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.129 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.129 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382892+0000", 2024-04-26T04:33:32.129 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8299999999999998e-07 2024-04-26T04:33:32.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.129 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.129 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.129 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382899+0000", 2024-04-26T04:33:32.129 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5009999999999998e-06 2024-04-26T04:33:32.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.130 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.130 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.130 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382905+0000", 2024-04-26T04:33:32.130 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.241e-06 2024-04-26T04:33:32.130 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.130 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.130 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.130 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.382910+0000", 2024-04-26T04:33:32.130 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.0289999999999996e-06 2024-04-26T04:33:32.130 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.130 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.130 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.131 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.596801+0000", 2024-04-26T04:33:32.131 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.213890254 2024-04-26T04:33:32.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.131 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.131 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:32.131 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.596847+0000", 2024-04-26T04:33:32.131 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.6711000000000001e-05 2024-04-26T04:33:32.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.131 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.131 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.131 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.994769+0000", 2024-04-26T04:33:32.131 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.39792192399999998 2024-04-26T04:33:32.131 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.132 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.132 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.132 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.994911+0000", 2024-04-26T04:33:32.132 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014135800000000001 2024-04-26T04:33:32.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.132 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.132 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.132 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:08.197242+0000", 2024-04-26T04:33:32.132 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20233146799999999 2024-04-26T04:33:32.132 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.132 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.132 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:841 3.fs0 3:f0b18d84:::smithi17829308-36:head [append 0~573440] snapc 16=[16,15,14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:09.300234+0000", 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 22.725966325999998, 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.52213421199999999, 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 841 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.133 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.134 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.134 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.134 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.300234+0000", 2024-04-26T04:33:32.134 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.134 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.134 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.134 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.300234+0000", 2024-04-26T04:33:32.134 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.134 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.134 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.135 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.300235+0000", 2024-04-26T04:33:32.135 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.7770000000000001e-06 2024-04-26T04:33:32.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.135 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.135 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.135 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.300876+0000", 2024-04-26T04:33:32.135 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00064050999999999995 2024-04-26T04:33:32.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.135 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.135 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.135 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.300902+0000", 2024-04-26T04:33:32.135 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6333e-05 2024-04-26T04:33:32.135 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.300921+0000", 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8899999999999999e-05 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.301110+0000", 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000189321 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.301278+0000", 2024-04-26T04:33:32.136 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000167913 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.302675+0000", 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0013964769999999999 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.304424+0000", 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001749401 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.137 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.822251+0000", 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51782681100000005 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.822368+0000", 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011676900000000001 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:842 3.fs0 3:f0b18d84:::smithi17829308-36:head [append 0~753664] snapc 16=[16,15,14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:09.312604+0000", 2024-04-26T04:33:32.138 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 22.713595855000001, 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51013875799999997, 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 842 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.312604+0000", 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.139 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.312604+0000", 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.312606+0000", 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.2630000000000002e-06 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.313439+0000", 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00083306799999999996 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.140 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.141 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.141 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.313473+0000", 2024-04-26T04:33:32.141 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3572999999999999e-05 2024-04-26T04:33:32.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.141 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.141 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.313496+0000", 2024-04-26T04:33:32.141 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3019000000000001e-05 2024-04-26T04:33:32.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.141 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.141 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.141 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.313642+0000", 2024-04-26T04:33:32.141 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014585800000000001 2024-04-26T04:33:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.313805+0000", 2024-04-26T04:33:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00016319199999999999 2024-04-26T04:33:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.315376+0000", 2024-04-26T04:33:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001570833 2024-04-26T04:33:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.142 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.318746+0000", 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0033702599999999999 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.822552+0000", 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50380583899999998 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.822743+0000", 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000190853 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.143 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:844 3.fs0 3:f0b18d84:::smithi17829308-36:head [setxattr _header (57) in=7b] snapc 16=[16,15,14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:09.316573+0000", 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 22.709626663000002, 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50695738099999998, 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.144 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 844 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.316573+0000", 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.316573+0000", 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.145 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.316575+0000", 2024-04-26T04:33:32.146 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.553e-06 2024-04-26T04:33:32.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.146 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.146 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.146 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.316583+0000", 2024-04-26T04:33:32.146 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.3429999999999996e-06 2024-04-26T04:33:32.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.146 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.146 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.146 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.316604+0000", 2024-04-26T04:33:32.146 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0806999999999999e-05 2024-04-26T04:33:32.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.146 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.416764+0000", 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100160148 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.517168+0000", 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10040428699999999 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.517342+0000", 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000173294 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.147 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.518090+0000", 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00074871499999999995 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.621319+0000", 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10322877699999999 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.823390+0000", 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20207059199999999 2024-04-26T04:33:32.148 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.149 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.149 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.149 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.823531+0000", 2024-04-26T04:33:32.149 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000140865 2024-04-26T04:33:32.149 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.149 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.149 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.149 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.149 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.149 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:32.151 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-26T04:33:32.278 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:32.278 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:32.278 INFO:teuthology.orchestra.run.smithi178.stdout: "num_ops": 0 2024-04-26T04:33:32.278 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:32.293 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-26T04:33:32.417 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:32.417 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [], 2024-04-26T04:33:32.417 INFO:teuthology.orchestra.run.smithi178.stdout: "complaint_time": 30, 2024-04-26T04:33:32.417 INFO:teuthology.orchestra.run.smithi178.stdout: "num_blocked_ops": 0 2024-04-26T04:33:32.417 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:32.430 INFO:teuthology.orchestra.run.smithi012.stderr:noscrub is unset 2024-04-26T04:33:32.431 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.0 to 1 (10000) 2024-04-26T04:33:32.432 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-26T04:33:32.461 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-26T04:33:32.462 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-04-26T04:33:32.557 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-04-26T04:33:32.557 INFO:teuthology.orchestra.run.smithi178.stdout: "size": 20, 2024-04-26T04:33:32.557 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 600, 2024-04-26T04:33:32.557 INFO:teuthology.orchestra.run.smithi178.stdout: "ops": [ 2024-04-26T04:33:32.557 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.557 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:21 3.5s0 3:a4a2788c:::smithi17829308-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:32.557 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:33:32.557 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 67.250622922000005, 2024-04-26T04:33:32.557 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71502945500000004, 2024-04-26T04:33:32.557 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.557 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 21 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.558 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304623+0000", 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304624+0000", 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.186e-06 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304629+0000", 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.3630000000000004e-06 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.559 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.560 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.560 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304643+0000", 2024-04-26T04:33:32.560 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4800000000000001e-05 2024-04-26T04:33:32.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.560 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.560 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.560 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304654+0000", 2024-04-26T04:33:32.560 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0665999999999999e-05 2024-04-26T04:33:32.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.560 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.560 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.560 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507573+0000", 2024-04-26T04:33:32.560 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20291931099999999 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.507644+0000", 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1131999999999996e-05 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.816490+0000", 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30884542700000001 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.816596+0000", 2024-04-26T04:33:32.561 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000105678 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.019652+0000", 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20305689199999999 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:25 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~147456,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 67.250498194000002, 2024-04-26T04:33:32.562 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61210120899999998, 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 25 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.563 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304748+0000", 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304749+0000", 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.277e-06 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304925+0000", 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017555399999999999 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.564 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.565 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.565 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304943+0000", 2024-04-26T04:33:32.565 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8529000000000002e-05 2024-04-26T04:33:32.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304954+0000", 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0879999999999999e-05 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608582+0000", 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30362831899999998 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.608711+0000", 2024-04-26T04:33:32.566 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012887 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609284+0000", 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00057331000000000005 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.710054+0000", 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100769761 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.916732+0000", 2024-04-26T04:33:32.567 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20667817799999999 2024-04-26T04:33:32.568 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.568 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.568 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.916849+0000", 2024-04-26T04:33:32.568 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000116531 2024-04-26T04:33:32.568 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.568 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.568 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.568 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.568 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:26 3.5s0 3:afb9923e:::smithi17829308-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-04-26T04:33:32.568 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 67.250276392000004, 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.71271566600000003, 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 26 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:33:32.569 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304970+0000", 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.2399999999999996e-07 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.570 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304975+0000", 2024-04-26T04:33:32.571 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.6460000000000001e-06 2024-04-26T04:33:32.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.571 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.571 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304984+0000", 2024-04-26T04:33:32.571 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.1540000000000008e-06 2024-04-26T04:33:32.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.571 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.571 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.304992+0000", 2024-04-26T04:33:32.571 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5259999999999996e-06 2024-04-26T04:33:32.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.571 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.572 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.572 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609850+0000", 2024-04-26T04:33:32.572 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.304857769 2024-04-26T04:33:32.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.572 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.572 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.609963+0000", 2024-04-26T04:33:32.572 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011346899999999999 2024-04-26T04:33:32.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.572 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.572 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.572 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.610506+0000", 2024-04-26T04:33:32.572 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00054322400000000003 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:25.710058+0000", 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.099551985999999995 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017545+0000", 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30748642599999998 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:26.017685+0000", 2024-04-26T04:33:32.573 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014054200000000001 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:360 3.5s0 3:a44e1dec:::smithi17829308-26:head [rollback 2] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.340709+0000", 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 57.214536967999997, 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.54345505299999997, 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.574 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 360 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340709+0000", 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340709+0000", 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340710+0000", 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9100000000000003e-07 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340714+0000", 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.1760000000000003e-06 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340725+0000", 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1664e-05 2024-04-26T04:33:32.576 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340734+0000", 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8899999999999996e-06 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340844+0000", 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000110005 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340851+0000", 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.7830000000000001e-06 2024-04-26T04:33:32.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.577334+0000", 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23648275699999999 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.577565+0000", 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00023120000000000001 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579037+0000", 2024-04-26T04:33:32.578 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00147218 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.682611+0000", 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103574176 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.884048+0000", 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201436371 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.884164+0000", 2024-04-26T04:33:32.579 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00011616 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:361 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~0] snapc 5=[5,2] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:35.340770+0000", 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 57.214475747999998, 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.644132284, 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.580 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 361 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340770+0000", 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340770+0000", 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.581 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340771+0000", 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.8400000000000004e-07 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340774+0000", 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.349e-06 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340783+0000", 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.3470000000000005e-06 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.582 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340792+0000", 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.8740000000000001e-06 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340874+0000", 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.1458000000000005e-05 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.340879+0000", 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4110000000000004e-06 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.583 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.584 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.579882+0000", 2024-04-26T04:33:32.584 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.23900315899999999 2024-04-26T04:33:32.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.584 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.584 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.580145+0000", 2024-04-26T04:33:32.584 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026314300000000002 2024-04-26T04:33:32.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.584 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.581275+0000", 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001129997 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.682889+0000", 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101613946 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.984790+0000", 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30190081600000002 2024-04-26T04:33:32.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:35.984902+0000", 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0001122 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:478 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~688128] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.219546+0000", 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.335699607999999, 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.60491518499999997, 2024-04-26T04:33:32.586 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.587 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.587 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.587 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.587 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.587 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.587 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 478 2024-04-26T04:33:32.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.587 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.587 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.587 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.587 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219546+0000", 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219546+0000", 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219549+0000", 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.322e-06 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219983+0000", 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00043452399999999999 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.219998+0000", 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5231999999999999e-05 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.220007+0000", 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9379999999999998e-06 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.590 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.320021+0000", 2024-04-26T04:33:32.590 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100013729 2024-04-26T04:33:32.590 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.320111+0000", 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 8.9907999999999994e-05 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.320914+0000", 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00080305000000000003 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427530+0000", 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.106616091 2024-04-26T04:33:32.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.592 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.592 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.824390+0000", 2024-04-26T04:33:32.592 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.396859621 2024-04-26T04:33:32.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.592 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.592 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.592 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.824461+0000", 2024-04-26T04:33:32.592 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1769999999999999e-05 2024-04-26T04:33:32.592 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.592 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.593 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.593 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.593 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:479 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~81920] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:32.593 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.221012+0000", 2024-04-26T04:33:32.593 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.334234256000002, 2024-04-26T04:33:32.593 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.70366377099999999, 2024-04-26T04:33:32.594 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.594 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.594 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.594 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.594 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 479 2024-04-26T04:33:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221012+0000", 2024-04-26T04:33:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.595 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221012+0000", 2024-04-26T04:33:32.596 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.596 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.596 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221013+0000", 2024-04-26T04:33:32.596 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4750000000000001e-06 2024-04-26T04:33:32.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.596 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.596 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.596 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221057+0000", 2024-04-26T04:33:32.597 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.4305999999999999e-05 2024-04-26T04:33:32.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.597 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.597 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221069+0000", 2024-04-26T04:33:32.597 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1783000000000001e-05 2024-04-26T04:33:32.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.597 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.597 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221077+0000", 2024-04-26T04:33:32.597 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7649999999999993e-06 2024-04-26T04:33:32.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.597 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321785+0000", 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100707597 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.322104+0000", 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00031923599999999998 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.323441+0000", 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001337233 2024-04-26T04:33:32.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427532+0000", 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10409099199999999 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924579+0000", 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49704692 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924675+0000", 2024-04-26T04:33:32.599 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6464000000000003e-05 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:480 3.5s0 3:afb9923e:::smithi17829308-9:head [setxattr _header (56) in=7b] snapc d=[c,b,a] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.334156366999999, 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.70383227299999995, 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.600 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 480 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.601 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221090+0000", 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.9999999999999997e-07 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221093+0000", 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.241e-06 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221100+0000", 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.6080000000000001e-06 2024-04-26T04:33:32.602 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.603 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.603 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221106+0000", 2024-04-26T04:33:32.603 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.4999999999999999e-06 2024-04-26T04:33:32.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.603 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.603 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.324423+0000", 2024-04-26T04:33:32.603 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.103317706 2024-04-26T04:33:32.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.603 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.603 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.603 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.324652+0000", 2024-04-26T04:33:32.604 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00022915899999999999 2024-04-26T04:33:32.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.604 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.604 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.325569+0000", 2024-04-26T04:33:32.604 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000916941 2024-04-26T04:33:32.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.604 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.604 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.427565+0000", 2024-04-26T04:33:32.604 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101995135 2024-04-26T04:33:32.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924820+0000", 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.49725566500000001 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.924922+0000", 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000101618 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.605 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:481 3.5s0 3:afb9923e:::smithi17829308-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-26T04:33:32.606 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:32:50.221113+0000", 2024-04-26T04:33:32.606 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 42.334132594000003, 2024-04-26T04:33:32.606 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.80581624500000004, 2024-04-26T04:33:32.606 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.606 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.606 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:32.606 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.610 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.610 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.610 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 481 2024-04-26T04:33:32.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.610 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221113+0000", 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221113+0000", 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221114+0000", 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 5.5700000000000002e-07 2024-04-26T04:33:32.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.612 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.612 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221117+0000", 2024-04-26T04:33:32.612 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.552e-06 2024-04-26T04:33:32.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.612 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.612 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.221125+0000", 2024-04-26T04:33:32.612 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.7710000000000006e-06 2024-04-26T04:33:32.612 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.612 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.612 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.612 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.321214+0000", 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100088539 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.326595+0000", 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0053815479999999999 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.326744+0000", 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014871700000000001 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.613 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.614 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.925438+0000", 2024-04-26T04:33:32.614 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.59869437999999997 2024-04-26T04:33:32.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.614 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.614 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:50.925572+0000", 2024-04-26T04:33:32.614 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000133912 2024-04-26T04:33:32.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.614 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.614 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.614 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:32:51.026930+0000", 2024-04-26T04:33:32.614 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.101357269 2024-04-26T04:33:32.614 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:728 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~475136] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.306711+0000", 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 30.248535386, 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.90587526100000004, 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.615 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 728 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.306711+0000", 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.306711+0000", 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.616 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.617 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.617 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.306713+0000", 2024-04-26T04:33:32.617 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.8389999999999998e-06 2024-04-26T04:33:32.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.617 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.617 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.307016+0000", 2024-04-26T04:33:32.617 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00030270699999999999 2024-04-26T04:33:32.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.617 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.618 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.618 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.307050+0000", 2024-04-26T04:33:32.618 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.4099e-05 2024-04-26T04:33:32.618 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.618 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.618 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.618 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.307069+0000", 2024-04-26T04:33:32.618 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.836e-05 2024-04-26T04:33:32.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.619 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.619 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396493+0000", 2024-04-26T04:33:32.619 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.089424218999999999 2024-04-26T04:33:32.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.619 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.619 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "op must wait for map", 2024-04-26T04:33:32.619 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396500+0000", 2024-04-26T04:33:32.619 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.6960000000000005e-06 2024-04-26T04:33:32.620 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.620 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.620 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.597275+0000", 2024-04-26T04:33:32.620 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20077457100000001 2024-04-26T04:33:32.620 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.620 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.620 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.621 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.597440+0000", 2024-04-26T04:33:32.621 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000165021 2024-04-26T04:33:32.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.621 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.621 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.621 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.599124+0000", 2024-04-26T04:33:32.622 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001683882 2024-04-26T04:33:32.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.622 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.622 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.622 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703777+0000", 2024-04-26T04:33:32.622 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.104652914 2024-04-26T04:33:32.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.623 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.623 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.212458+0000", 2024-04-26T04:33:32.623 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50868092200000004 2024-04-26T04:33:32.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.623 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.624 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.624 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.212586+0000", 2024-04-26T04:33:32.624 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012803099999999999 2024-04-26T04:33:32.624 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.624 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.624 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.624 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.625 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.625 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:729 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~655360] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:32.625 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.316292+0000", 2024-04-26T04:33:32.625 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 30.238953424000002, 2024-04-26T04:33:32.625 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.89658193900000005, 2024-04-26T04:33:32.625 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.626 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.626 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.626 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.626 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.626 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.626 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 729 2024-04-26T04:33:32.627 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.627 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.627 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.627 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.627 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316292+0000", 2024-04-26T04:33:32.627 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.628 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.628 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.628 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.628 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316292+0000", 2024-04-26T04:33:32.628 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.628 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.629 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.629 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.629 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316295+0000", 2024-04-26T04:33:32.629 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.4490000000000002e-06 2024-04-26T04:33:32.629 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.629 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.629 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.630 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316680+0000", 2024-04-26T04:33:32.630 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00038542699999999998 2024-04-26T04:33:32.630 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.630 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.630 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.630 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316716+0000", 2024-04-26T04:33:32.631 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.5405999999999999e-05 2024-04-26T04:33:32.631 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.631 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.631 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.631 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.316736+0000", 2024-04-26T04:33:32.631 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.0505000000000001e-05 2024-04-26T04:33:32.632 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.632 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.632 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396572+0000", 2024-04-26T04:33:32.632 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.079835357999999995 2024-04-26T04:33:32.632 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.632 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.633 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.633 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396575+0000", 2024-04-26T04:33:32.633 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.529e-06 2024-04-26T04:33:32.633 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.633 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.633 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.634 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.600521+0000", 2024-04-26T04:33:32.634 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20394542500000001 2024-04-26T04:33:32.634 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.634 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.634 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.634 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.600677+0000", 2024-04-26T04:33:32.635 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00015607699999999999 2024-04-26T04:33:32.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.635 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.635 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.635 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.602205+0000", 2024-04-26T04:33:32.635 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001528661 2024-04-26T04:33:32.636 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.636 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.636 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703787+0000", 2024-04-26T04:33:32.636 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10158194500000001 2024-04-26T04:33:32.636 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.636 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.637 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.637 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.212771+0000", 2024-04-26T04:33:32.637 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50898380499999996 2024-04-26T04:33:32.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.637 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.637 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.638 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.212874+0000", 2024-04-26T04:33:32.638 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000103352 2024-04-26T04:33:32.638 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.638 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.638 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.638 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.639 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.639 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:730 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~622592] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:32.639 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.325217+0000", 2024-04-26T04:33:32.639 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 30.230029382000001, 2024-04-26T04:33:32.639 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.88791738899999995, 2024-04-26T04:33:32.640 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.640 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.640 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.640 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.640 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.640 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.641 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 730 2024-04-26T04:33:32.641 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.641 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.641 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.641 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.641 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325217+0000", 2024-04-26T04:33:32.642 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.642 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.642 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.642 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325217+0000", 2024-04-26T04:33:32.642 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.643 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.643 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.643 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.643 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325219+0000", 2024-04-26T04:33:32.643 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.283e-06 2024-04-26T04:33:32.643 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.644 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.644 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.644 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325555+0000", 2024-04-26T04:33:32.644 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00033651699999999999 2024-04-26T04:33:32.644 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.644 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.644 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.645 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325585+0000", 2024-04-26T04:33:32.645 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9173999999999999e-05 2024-04-26T04:33:32.645 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.645 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.645 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.645 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.325601+0000", 2024-04-26T04:33:32.645 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6551000000000001e-05 2024-04-26T04:33:32.645 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.645 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.645 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.646 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396592+0000", 2024-04-26T04:33:32.646 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.070990756000000002 2024-04-26T04:33:32.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.646 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.646 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.646 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396595+0000", 2024-04-26T04:33:32.646 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.061e-06 2024-04-26T04:33:32.646 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.647 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.647 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.603348+0000", 2024-04-26T04:33:32.647 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.206753411 2024-04-26T04:33:32.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.647 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.647 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.647 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.603532+0000", 2024-04-26T04:33:32.648 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00018416400000000001 2024-04-26T04:33:32.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.648 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.648 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.648 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.605092+0000", 2024-04-26T04:33:32.648 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.001559337 2024-04-26T04:33:32.648 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.648 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.648 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.649 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703790+0000", 2024-04-26T04:33:32.649 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.098698498999999995 2024-04-26T04:33:32.649 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.649 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.649 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.649 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213032+0000", 2024-04-26T04:33:32.649 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50924209399999998 2024-04-26T04:33:32.649 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.649 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.650 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.650 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213134+0000", 2024-04-26T04:33:32.650 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000101542 2024-04-26T04:33:32.650 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.650 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.650 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.650 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:731 3.5s0 3:a44e1dec:::smithi17829308-26:head [append 0~278528] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:32.650 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.329197+0000", 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 30.226048433999999, 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.88429409199999998, 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 731 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329197+0000", 2024-04-26T04:33:32.651 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.652 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.652 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329197+0000", 2024-04-26T04:33:32.652 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.652 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.652 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329199+0000", 2024-04-26T04:33:32.652 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.5910000000000001e-06 2024-04-26T04:33:32.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.652 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329372+0000", 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00017300599999999999 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329398+0000", 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.6400000000000001e-05 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329413+0000", 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4548e-05 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.653 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.654 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396610+0000", 2024-04-26T04:33:32.654 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.067197398000000005 2024-04-26T04:33:32.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.654 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.654 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.654 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.396613+0000", 2024-04-26T04:33:32.654 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.033e-06 2024-04-26T04:33:32.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.654 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.654 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.654 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.606931+0000", 2024-04-26T04:33:32.654 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21031707599999999 2024-04-26T04:33:32.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.607074+0000", 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000143144 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.608197+0000", 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0011229980000000001 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703793+0000", 2024-04-26T04:33:32.655 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.095596134999999999 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213368+0000", 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50957523800000004 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213492+0000", 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00012352500000000001 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.656 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:732 3.5s0 3:a44e1dec:::smithi17829308-26:head [setxattr _header (57) in=7b] snapc 11=[11,10,e,b] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.329456+0000", 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 30.225789777999999, 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.8844417, 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 732 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.657 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329456+0000", 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329456+0000", 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329457+0000", 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1969999999999999e-06 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.658 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329465+0000", 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.1969999999999999e-06 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329483+0000", 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.8351000000000001e-05 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329497+0000", 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.4114000000000001e-05 2024-04-26T04:33:32.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.407278+0000", 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.077780571000000007 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.407289+0000", 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.111e-05 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.609056+0000", 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201767264 2024-04-26T04:33:32.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.609198+0000", 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014221700000000001 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.609956+0000", 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00075800100000000001 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703800+0000", 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.093843736999999997 2024-04-26T04:33:32.661 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213750+0000", 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.50995032600000001 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.213898+0000", 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000147615 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.662 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:733 3.5s0 3:a44e1dec:::smithi17829308-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:02.329515+0000", 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 30.225730882000001, 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0903875629999999, 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 733 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.663 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329515+0000", 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329515+0000", 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329516+0000", 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.2299999999999999e-07 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.664 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.665 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.665 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329520+0000", 2024-04-26T04:33:32.665 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.5399999999999997e-06 2024-04-26T04:33:32.665 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.665 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.665 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329537+0000", 2024-04-26T04:33:32.665 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.6393999999999999e-05 2024-04-26T04:33:32.665 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.665 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.665 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.665 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.329550+0000", 2024-04-26T04:33:32.666 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2934e-05 2024-04-26T04:33:32.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.666 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.666 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.666 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.407334+0000", 2024-04-26T04:33:32.666 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.077784331999999998 2024-04-26T04:33:32.666 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.667 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.667 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.407342+0000", 2024-04-26T04:33:32.667 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.9699999999999999e-06 2024-04-26T04:33:32.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.667 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.667 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703462+0000", 2024-04-26T04:33:32.668 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.29612015899999999 2024-04-26T04:33:32.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.668 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.668 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:32.668 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:02.703734+0000", 2024-04-26T04:33:32.668 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00027200199999999998 2024-04-26T04:33:32.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.669 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.669 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.214603+0000", 2024-04-26T04:33:32.669 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.51086900899999999 2024-04-26T04:33:32.669 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.669 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.670 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.214847+0000", 2024-04-26T04:33:32.670 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00024385300000000001 2024-04-26T04:33:32.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.670 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.670 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.670 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.419903+0000", 2024-04-26T04:33:32.671 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.205055447 2024-04-26T04:33:32.671 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.671 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.671 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.671 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.671 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(osd.5.23:4 3.5s0 3:afb9923e:::smithi17829308-9:head [assert-version v44,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e68)", 2024-04-26T04:33:32.671 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:03.630782+0000", 2024-04-26T04:33:32.672 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 28.924464387, 2024-04-26T04:33:32.672 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.60668015399999997, 2024-04-26T04:33:32.672 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.672 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.672 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:32.672 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.672 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.673 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.673 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630782+0000", 2024-04-26T04:33:32.673 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.673 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.673 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.673 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630782+0000", 2024-04-26T04:33:32.673 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.674 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.674 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.674 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630785+0000", 2024-04-26T04:33:32.674 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3270000000000001e-06 2024-04-26T04:33:32.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.675 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.675 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.675 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630798+0000", 2024-04-26T04:33:32.675 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.2893e-05 2024-04-26T04:33:32.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.675 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.675 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.676 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630832+0000", 2024-04-26T04:33:32.676 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.3865999999999997e-05 2024-04-26T04:33:32.676 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.676 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.676 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.676 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.630853+0000", 2024-04-26T04:33:32.676 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.1437000000000002e-05 2024-04-26T04:33:32.676 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.677 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.677 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.677 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.833963+0000", 2024-04-26T04:33:32.677 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.20310972999999999 2024-04-26T04:33:32.677 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.677 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.677 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.677 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:03.834253+0000", 2024-04-26T04:33:32.677 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00029035900000000002 2024-04-26T04:33:32.677 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.678 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.678 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.678 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:04.237462+0000", 2024-04-26T04:33:32.678 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.403208542 2024-04-26T04:33:32.678 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.678 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.678 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.678 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.678 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:813 3.5s0 3:ac36f45f:::smithi17829308-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-04-26T04:33:32.678 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:07.365605+0000", 2024-04-26T04:33:32.678 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 25.18964051, 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.56179527200000001, 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 813 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365605+0000", 2024-04-26T04:33:32.679 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.680 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.680 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.680 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.680 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365605+0000", 2024-04-26T04:33:32.680 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.680 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.680 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.680 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.680 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365606+0000", 2024-04-26T04:33:32.680 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9499999999999998e-07 2024-04-26T04:33:32.680 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.680 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365607+0000", 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.339e-06 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365612+0000", 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.532e-06 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.365615+0000", 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 3.191e-06 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.681 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.682 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.682 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.366700+0000", 2024-04-26T04:33:32.682 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0010852710000000001 2024-04-26T04:33:32.682 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.682 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.682 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.682 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.366706+0000", 2024-04-26T04:33:32.682 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 6.2230000000000004e-06 2024-04-26T04:33:32.682 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.682 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.682 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.682 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580054+0000", 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.21334779800000001 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.580129+0000", 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5333000000000005e-05 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.884613+0000", 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30448329899999999 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.683 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.884873+0000", 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00026001900000000001 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:07.927401+0000", 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.042527971999999997 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:834 3.5s0 3:afb9923e:::smithi17829308-9:head [append 0~0] snapc 16=[16,15,14,13,12,11,10] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:09.269198+0000", 2024-04-26T04:33:32.684 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 23.286048314999999, 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.610278235, 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 834 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269198+0000", 2024-04-26T04:33:32.685 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.686 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.686 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.686 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.686 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269198+0000", 2024-04-26T04:33:32.686 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.686 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.686 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.686 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.686 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269200+0000", 2024-04-26T04:33:32.686 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3159999999999999e-06 2024-04-26T04:33:32.686 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.687 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.687 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.687 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269211+0000", 2024-04-26T04:33:32.687 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.1487e-05 2024-04-26T04:33:32.687 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.687 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.687 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.687 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269235+0000", 2024-04-26T04:33:32.688 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.3476999999999999e-05 2024-04-26T04:33:32.688 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.688 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.688 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.688 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.269254+0000", 2024-04-26T04:33:32.688 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9219e-05 2024-04-26T04:33:32.689 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.689 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.689 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.689 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.273442+0000", 2024-04-26T04:33:32.689 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.0041875920000000004 2024-04-26T04:33:32.689 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.689 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.689 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting_for_map not empty", 2024-04-26T04:33:32.689 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.273461+0000", 2024-04-26T04:33:32.689 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.9656999999999998e-05 2024-04-26T04:33:32.689 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.689 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.690 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.690 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.575901+0000", 2024-04-26T04:33:32.690 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30243927799999998 2024-04-26T04:33:32.690 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.690 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.690 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.690 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.576065+0000", 2024-04-26T04:33:32.690 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000164864 2024-04-26T04:33:32.690 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.691 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.691 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.691 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.576829+0000", 2024-04-26T04:33:32.691 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00076318299999999996 2024-04-26T04:33:32.691 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.691 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.691 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.691 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.677695+0000", 2024-04-26T04:33:32.691 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.100866385 2024-04-26T04:33:32.691 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.691 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.691 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.692 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.879354+0000", 2024-04-26T04:33:32.692 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.201658688 2024-04-26T04:33:32.692 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.692 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.692 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.692 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:09.879476+0000", 2024-04-26T04:33:32.692 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.000122089 2024-04-26T04:33:32.692 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.692 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.692 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.692 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.692 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:954 3.5s0 3:a5af5236:::smithi17829308-45:head [setxattr _header (57) in=7b] snapc 18=[16] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:19.398965+0000", 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 13.156281368, 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.61122250099999997, 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 954 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.693 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398965+0000", 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398965+0000", 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398965+0000", 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.33e-07 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.694 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398970+0000", 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 4.4669999999999998e-06 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398980+0000", 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 1.0003000000000001e-05 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398988+0000", 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.5989999999999996e-06 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.695 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.503122+0000", 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.104134428 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.503405+0000", 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00028289800000000001 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_started", 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.504791+0000", 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00138569 2024-04-26T04:33:32.696 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "sub_op_committed", 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.605750+0000", 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.10095949999999999 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "commit_sent", 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:20.010040+0000", 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.40428993299999999 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:20.010187+0000", 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00014705 2024-04-26T04:33:32.697 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: "description": "osd_op(client.4670.0:955 3.5s0 3:a5af5236:::smithi17829308-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e84)", 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: "initiated_at": "2024-04-26T04:33:19.398998+0000", 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: "age": 13.156247631999999, 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.91489952299999999, 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: "continuous": false, 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: "type_data": { 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: "flag_point": "started", 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: "client_info": { 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: "client": "client.4670", 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: "client_addr": "172.21.15.178:0/239274625", 2024-04-26T04:33:32.698 INFO:teuthology.orchestra.run.smithi178.stdout: "tid": 955 2024-04-26T04:33:32.699 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.699 INFO:teuthology.orchestra.run.smithi178.stdout: "events": [ 2024-04-26T04:33:32.699 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.699 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "initiated", 2024-04-26T04:33:32.699 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398998+0000", 2024-04-26T04:33:32.699 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.699 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.699 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.699 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "header_read", 2024-04-26T04:33:32.699 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398998+0000", 2024-04-26T04:33:32.699 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0 2024-04-26T04:33:32.699 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.699 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "throttled", 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.398999+0000", 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 7.9599999999999998e-07 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "all_read", 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.399002+0000", 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 2.9440000000000001e-06 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "dispatched", 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.399012+0000", 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.9259999999999995e-06 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.700 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "queued_for_pg", 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.399022+0000", 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 9.6330000000000008e-06 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.706787+0000", 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30776545599999999 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "waiting for rw locks", 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:19.707088+0000", 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00030146399999999999 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.701 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.702 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "reached_pg", 2024-04-26T04:33:32.702 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:20.010863+0000", 2024-04-26T04:33:32.702 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30377425800000002 2024-04-26T04:33:32.702 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.702 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.702 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "started", 2024-04-26T04:33:32.702 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:20.010993+0000", 2024-04-26T04:33:32.702 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.00013003400000000001 2024-04-26T04:33:32.702 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-04-26T04:33:32.702 INFO:teuthology.orchestra.run.smithi178.stdout: { 2024-04-26T04:33:32.702 INFO:teuthology.orchestra.run.smithi178.stdout: "event": "done", 2024-04-26T04:33:32.702 INFO:teuthology.orchestra.run.smithi178.stdout: "time": "2024-04-26T04:33:20.313898+0000", 2024-04-26T04:33:32.702 INFO:teuthology.orchestra.run.smithi178.stdout: "duration": 0.30290501199999997 2024-04-26T04:33:32.703 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.703 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.703 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.703 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-04-26T04:33:32.703 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-04-26T04:33:32.703 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-04-26T04:33:33.446 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-04-26T04:33:33.446 INFO:teuthology.orchestra.run.smithi012.stderr:nodeep-scrub is unset 2024-04-26T04:33:33.477 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-26T04:33:34.456 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.4 to 1 (10000) 2024-04-26T04:33:34.480 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-04-26T04:33:35.459 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-04-26T04:33:35.483 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-04-26T04:33:36.475 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.8 to 1 (10000) 2024-04-26T04:33:36.500 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 1 2024-04-26T04:33:37.476 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.8 primary-affinity to 1 (8655362) 2024-04-26T04:33:37.507 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-04-26T04:33:38.495 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.12 to 1 (10000) 2024-04-26T04:33:38.522 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 12 1 2024-04-26T04:33:39.503 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.12 primary-affinity to 1 (8655362) 2024-04-26T04:33:39.528 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-26T04:33:40.506 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.1 to 1 (10000) 2024-04-26T04:33:40.531 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-04-26T04:33:41.524 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-04-26T04:33:41.548 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-26T04:33:42.530 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.5 to 1 (10000) 2024-04-26T04:33:42.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-04-26T04:33:43.532 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-04-26T04:33:43.560 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-04-26T04:33:44.543 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.9 to 1 (10000) 2024-04-26T04:33:44.571 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 9 1 2024-04-26T04:33:45.552 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.9 primary-affinity to 1 (8655362) 2024-04-26T04:33:45.581 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-04-26T04:33:46.567 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.13 to 1 (10000) 2024-04-26T04:33:46.592 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 13 1 2024-04-26T04:33:47.570 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.13 primary-affinity to 1 (8655362) 2024-04-26T04:33:47.594 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-26T04:33:48.581 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.2 to 1 (10000) 2024-04-26T04:33:48.609 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-04-26T04:33:49.592 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-04-26T04:33:49.618 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-26T04:33:50.610 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.6 to 1 (10000) 2024-04-26T04:33:50.635 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-04-26T04:33:51.612 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-04-26T04:33:51.639 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-04-26T04:33:52.622 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.10 to 1 (10000) 2024-04-26T04:33:52.646 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 10 1 2024-04-26T04:33:53.630 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.10 primary-affinity to 1 (8655362) 2024-04-26T04:33:53.657 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-04-26T04:33:54.641 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.14 to 1 (10000) 2024-04-26T04:33:54.674 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 14 1 2024-04-26T04:33:55.651 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.14 primary-affinity to 1 (8655362) 2024-04-26T04:33:55.679 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-26T04:33:56.662 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.3 to 1 (10000) 2024-04-26T04:33:56.689 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-04-26T04:33:57.682 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-04-26T04:33:57.708 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-26T04:33:58.685 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.7 to 1 (10000) 2024-04-26T04:33:58.712 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-04-26T04:33:59.696 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-04-26T04:33:59.723 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-04-26T04:34:00.712 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.11 to 1 (10000) 2024-04-26T04:34:00.737 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 11 1 2024-04-26T04:34:01.714 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.11 primary-affinity to 1 (8655362) 2024-04-26T04:34:01.741 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-04-26T04:34:02.724 INFO:teuthology.orchestra.run.smithi012.stderr:reweighted osd.15 to 1 (10000) 2024-04-26T04:34:02.752 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 15 1 2024-04-26T04:34:03.734 INFO:teuthology.orchestra.run.smithi012.stderr:set osd.15 primary-affinity to 1 (8655362) 2024-04-26T04:34:03.760 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-04-26T04:34:03.761 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-04-26T04:34:03.761 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-04-26T04:34:03.761 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-04-26T04:34:03.761 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T04:34:03.762 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T04:34:04.071 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:34:04.072 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":128,"fsid":"12c27fbe-0161-48d0-88ec-42a226685437","created":"2024-04-26T04:31:56.451062+0000","modified":"2024-04-26T04:34:03.731364+0000","last_up_change":"2024-04-26T04:32:09.452464+0000","last_in_change":"2024-04-26T04:33:31.417237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:32:07.426875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-26T04:32:11.318957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"55a4f254-2c2a-4007-9c52-b89df6fffb69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6817","nonce":1754063127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6819","nonce":1754063127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6823","nonce":1754063127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6821","nonce":1754063127}]},"public_addr":"172.21.15.12:6817/1754063127","cluster_addr":"172.21.15.12:6819/1754063127","heartbeat_back_addr":"172.21.15.12:6823/1754063127","heartbeat_front_addr":"172.21.15.12:6821/1754063127","state":["exists","up"]},{"osd":1,"uuid":"32db4f97-c8ac-481c-9e80-18fc91004298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6809","nonce":3264326676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6811","nonce":3264326676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6817","nonce":3264326676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6813","nonce":3264326676}]},"public_addr":"172.21.15.113:6809/3264326676","cluster_addr":"172.21.15.113:6811/3264326676","heartbeat_back_addr":"172.21.15.113:6817/3264326676","heartbeat_front_addr":"172.21.15.113:6813/3264326676","state":["exists","up"]},{"osd":2,"uuid":"2cd26466-5d59-4599-ba3f-a78b0e3b681f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6802","nonce":1454475448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6806","nonce":1454475448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6814","nonce":1454475448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6810","nonce":1454475448}]},"public_addr":"172.21.15.137:6802/1454475448","cluster_addr":"172.21.15.137:6806/1454475448","heartbeat_back_addr":"172.21.15.137:6814/1454475448","heartbeat_front_addr":"172.21.15.137:6810/1454475448","state":["exists","up"]},{"osd":3,"uuid":"0945017d-9525-49d6-aa67-6635acd94c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6803","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6805","nonce":2898698985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6810","nonce":2898698985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6817","nonce":2898698985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6815","nonce":2898698985}]},"public_addr":"172.21.15.178:6805/2898698985","cluster_addr":"172.21.15.178:6810/2898698985","heartbeat_back_addr":"172.21.15.178:6817/2898698985","heartbeat_front_addr":"172.21.15.178:6815/2898698985","state":["exists","up"]},{"osd":4,"uuid":"38dc68bf-0a89-4900-9d35-24c70368447a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6809","nonce":3209840823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6811","nonce":3209840823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6815","nonce":3209840823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6813","nonce":3209840823}]},"public_addr":"172.21.15.12:6809/3209840823","cluster_addr":"172.21.15.12:6811/3209840823","heartbeat_back_addr":"172.21.15.12:6815/3209840823","heartbeat_front_addr":"172.21.15.12:6813/3209840823","state":["exists","up"]},{"osd":5,"uuid":"e91cc8a1-57e1-4760-a329-8a60aa752a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6801","nonce":1475637773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6803","nonce":1475637773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6807","nonce":1475637773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6805","nonce":1475637773}]},"public_addr":"172.21.15.113:6801/1475637773","cluster_addr":"172.21.15.113:6803/1475637773","heartbeat_back_addr":"172.21.15.113:6807/1475637773","heartbeat_front_addr":"172.21.15.113:6805/1475637773","state":["exists","up"]},{"osd":6,"uuid":"a1322130-3391-432d-a1ee-f83297885029","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6817","nonce":771338783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6819","nonce":771338783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6823","nonce":771338783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6821","nonce":771338783}]},"public_addr":"172.21.15.137:6817/771338783","cluster_addr":"172.21.15.137:6819/771338783","heartbeat_back_addr":"172.21.15.137:6823/771338783","heartbeat_front_addr":"172.21.15.137:6821/771338783","state":["exists","up"]},{"osd":7,"uuid":"02ee4ca2-9d48-4aff-9641-1823760c7a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6827","nonce":1017152077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6829","nonce":1017152077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6833","nonce":1017152077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6831","nonce":1017152077}]},"public_addr":"172.21.15.178:6827/1017152077","cluster_addr":"172.21.15.178:6829/1017152077","heartbeat_back_addr":"172.21.15.178:6833/1017152077","heartbeat_front_addr":"172.21.15.178:6831/1017152077","state":["exists","up"]},{"osd":8,"uuid":"ea3d5f62-3d68-4824-8556-da8db36553fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6801","nonce":2153129138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6803","nonce":2153129138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6807","nonce":2153129138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6805","nonce":2153129138}]},"public_addr":"172.21.15.12:6801/2153129138","cluster_addr":"172.21.15.12:6803/2153129138","heartbeat_back_addr":"172.21.15.12:6807/2153129138","heartbeat_front_addr":"172.21.15.12:6805/2153129138","state":["exists","up"]},{"osd":9,"uuid":"e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6825","nonce":359931551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6827","nonce":359931551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6831","nonce":359931551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6829","nonce":359931551}]},"public_addr":"172.21.15.113:6825/359931551","cluster_addr":"172.21.15.113:6827/359931551","heartbeat_back_addr":"172.21.15.113:6831/359931551","heartbeat_front_addr":"172.21.15.113:6829/359931551","state":["exists","up"]},{"osd":10,"uuid":"8c4f8df5-465a-486e-9413-1de14e33c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6801","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6803","nonce":1533480861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6807","nonce":1533480861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6815","nonce":1533480861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6811","nonce":1533480861}]},"public_addr":"172.21.15.137:6803/1533480861","cluster_addr":"172.21.15.137:6807/1533480861","heartbeat_back_addr":"172.21.15.137:6815/1533480861","heartbeat_front_addr":"172.21.15.137:6811/1533480861","state":["exists","up"]},{"osd":11,"uuid":"6fe54c2d-0b95-4235-8c22-51230cfaa591","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6819","nonce":1683682957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6821","nonce":1683682957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6825","nonce":1683682957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6823","nonce":1683682957}]},"public_addr":"172.21.15.178:6819/1683682957","cluster_addr":"172.21.15.178:6821/1683682957","heartbeat_back_addr":"172.21.15.178:6825/1683682957","heartbeat_front_addr":"172.21.15.178:6823/1683682957","state":["exists","up"]},{"osd":12,"uuid":"5ddd0f05-fb78-43bb-95ec-3fc7985603b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6825","nonce":2509311840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6827","nonce":2509311840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6831","nonce":2509311840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6829","nonce":2509311840}]},"public_addr":"172.21.15.12:6825/2509311840","cluster_addr":"172.21.15.12:6827/2509311840","heartbeat_back_addr":"172.21.15.12:6831/2509311840","heartbeat_front_addr":"172.21.15.12:6829/2509311840","state":["exists","up"]},{"osd":13,"uuid":"37eb42c1-f29e-49f5-b8f6-035178038e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6816","nonce":3716239663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6819","nonce":3716239663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6823","nonce":3716239663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6821","nonce":3716239663}]},"public_addr":"172.21.15.113:6816/3716239663","cluster_addr":"172.21.15.113:6819/3716239663","heartbeat_back_addr":"172.21.15.113:6823/3716239663","heartbeat_front_addr":"172.21.15.113:6821/3716239663","state":["exists","up"]},{"osd":14,"uuid":"b2267367-a857-4a5c-a8d1-6a9b23c748f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6825","nonce":1454956132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6827","nonce":1454956132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6831","nonce":1454956132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6829","nonce":1454956132}]},"public_addr":"172.21.15.137:6825/1454956132","cluster_addr":"172.21.15.137:6827/1454956132","heartbeat_back_addr":"172.21.15.137:6831/1454956132","heartbeat_front_addr":"172.21.15.137:6829/1454956132","state":["exists","up"]},{"osd":15,"uuid":"9933b5aa-e091-43b1-b788-5b79695bab11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6804","nonce":314165740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6807","nonce":314165740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6814","nonce":314165740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6809","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6811","nonce":314165740}]},"public_addr":"172.21.15.178:6804/314165740","cluster_addr":"172.21.15.178:6807/314165740","heartbeat_back_addr":"172.21.15.178:6814/314165740","heartbeat_front_addr":"172.21.15.178:6811/314165740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:30.569382+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:09.218081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:08.352200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:13.912651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:36.408970+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:33.547355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:28.141489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:25.153786+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:26.506163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:10.178898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:20.633419+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:32.266862+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:19.550387+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:10.407663+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:32.078820+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:14.071867+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":20,"length":1},{"begin":22,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:34:04.091 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T04:34:04.091 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T04:34:04.405 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:34:04.406 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":128,"fsid":"12c27fbe-0161-48d0-88ec-42a226685437","created":"2024-04-26T04:31:56.451062+0000","modified":"2024-04-26T04:34:03.731364+0000","last_up_change":"2024-04-26T04:32:09.452464+0000","last_in_change":"2024-04-26T04:33:31.417237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:32:07.426875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-26T04:32:11.318957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"55a4f254-2c2a-4007-9c52-b89df6fffb69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6817","nonce":1754063127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6819","nonce":1754063127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6823","nonce":1754063127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6821","nonce":1754063127}]},"public_addr":"172.21.15.12:6817/1754063127","cluster_addr":"172.21.15.12:6819/1754063127","heartbeat_back_addr":"172.21.15.12:6823/1754063127","heartbeat_front_addr":"172.21.15.12:6821/1754063127","state":["exists","up"]},{"osd":1,"uuid":"32db4f97-c8ac-481c-9e80-18fc91004298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6809","nonce":3264326676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6811","nonce":3264326676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6817","nonce":3264326676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6813","nonce":3264326676}]},"public_addr":"172.21.15.113:6809/3264326676","cluster_addr":"172.21.15.113:6811/3264326676","heartbeat_back_addr":"172.21.15.113:6817/3264326676","heartbeat_front_addr":"172.21.15.113:6813/3264326676","state":["exists","up"]},{"osd":2,"uuid":"2cd26466-5d59-4599-ba3f-a78b0e3b681f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6802","nonce":1454475448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6806","nonce":1454475448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6814","nonce":1454475448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6810","nonce":1454475448}]},"public_addr":"172.21.15.137:6802/1454475448","cluster_addr":"172.21.15.137:6806/1454475448","heartbeat_back_addr":"172.21.15.137:6814/1454475448","heartbeat_front_addr":"172.21.15.137:6810/1454475448","state":["exists","up"]},{"osd":3,"uuid":"0945017d-9525-49d6-aa67-6635acd94c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6803","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6805","nonce":2898698985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6810","nonce":2898698985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6817","nonce":2898698985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6815","nonce":2898698985}]},"public_addr":"172.21.15.178:6805/2898698985","cluster_addr":"172.21.15.178:6810/2898698985","heartbeat_back_addr":"172.21.15.178:6817/2898698985","heartbeat_front_addr":"172.21.15.178:6815/2898698985","state":["exists","up"]},{"osd":4,"uuid":"38dc68bf-0a89-4900-9d35-24c70368447a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6809","nonce":3209840823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6811","nonce":3209840823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6815","nonce":3209840823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6813","nonce":3209840823}]},"public_addr":"172.21.15.12:6809/3209840823","cluster_addr":"172.21.15.12:6811/3209840823","heartbeat_back_addr":"172.21.15.12:6815/3209840823","heartbeat_front_addr":"172.21.15.12:6813/3209840823","state":["exists","up"]},{"osd":5,"uuid":"e91cc8a1-57e1-4760-a329-8a60aa752a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6801","nonce":1475637773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6803","nonce":1475637773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6807","nonce":1475637773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6805","nonce":1475637773}]},"public_addr":"172.21.15.113:6801/1475637773","cluster_addr":"172.21.15.113:6803/1475637773","heartbeat_back_addr":"172.21.15.113:6807/1475637773","heartbeat_front_addr":"172.21.15.113:6805/1475637773","state":["exists","up"]},{"osd":6,"uuid":"a1322130-3391-432d-a1ee-f83297885029","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6817","nonce":771338783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6819","nonce":771338783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6823","nonce":771338783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6821","nonce":771338783}]},"public_addr":"172.21.15.137:6817/771338783","cluster_addr":"172.21.15.137:6819/771338783","heartbeat_back_addr":"172.21.15.137:6823/771338783","heartbeat_front_addr":"172.21.15.137:6821/771338783","state":["exists","up"]},{"osd":7,"uuid":"02ee4ca2-9d48-4aff-9641-1823760c7a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6827","nonce":1017152077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6829","nonce":1017152077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6833","nonce":1017152077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6831","nonce":1017152077}]},"public_addr":"172.21.15.178:6827/1017152077","cluster_addr":"172.21.15.178:6829/1017152077","heartbeat_back_addr":"172.21.15.178:6833/1017152077","heartbeat_front_addr":"172.21.15.178:6831/1017152077","state":["exists","up"]},{"osd":8,"uuid":"ea3d5f62-3d68-4824-8556-da8db36553fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6801","nonce":2153129138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6803","nonce":2153129138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6807","nonce":2153129138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6805","nonce":2153129138}]},"public_addr":"172.21.15.12:6801/2153129138","cluster_addr":"172.21.15.12:6803/2153129138","heartbeat_back_addr":"172.21.15.12:6807/2153129138","heartbeat_front_addr":"172.21.15.12:6805/2153129138","state":["exists","up"]},{"osd":9,"uuid":"e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6825","nonce":359931551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6827","nonce":359931551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6831","nonce":359931551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6829","nonce":359931551}]},"public_addr":"172.21.15.113:6825/359931551","cluster_addr":"172.21.15.113:6827/359931551","heartbeat_back_addr":"172.21.15.113:6831/359931551","heartbeat_front_addr":"172.21.15.113:6829/359931551","state":["exists","up"]},{"osd":10,"uuid":"8c4f8df5-465a-486e-9413-1de14e33c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6801","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6803","nonce":1533480861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6807","nonce":1533480861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6815","nonce":1533480861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6811","nonce":1533480861}]},"public_addr":"172.21.15.137:6803/1533480861","cluster_addr":"172.21.15.137:6807/1533480861","heartbeat_back_addr":"172.21.15.137:6815/1533480861","heartbeat_front_addr":"172.21.15.137:6811/1533480861","state":["exists","up"]},{"osd":11,"uuid":"6fe54c2d-0b95-4235-8c22-51230cfaa591","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6819","nonce":1683682957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6821","nonce":1683682957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6825","nonce":1683682957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6823","nonce":1683682957}]},"public_addr":"172.21.15.178:6819/1683682957","cluster_addr":"172.21.15.178:6821/1683682957","heartbeat_back_addr":"172.21.15.178:6825/1683682957","heartbeat_front_addr":"172.21.15.178:6823/1683682957","state":["exists","up"]},{"osd":12,"uuid":"5ddd0f05-fb78-43bb-95ec-3fc7985603b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6825","nonce":2509311840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6827","nonce":2509311840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6831","nonce":2509311840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6829","nonce":2509311840}]},"public_addr":"172.21.15.12:6825/2509311840","cluster_addr":"172.21.15.12:6827/2509311840","heartbeat_back_addr":"172.21.15.12:6831/2509311840","heartbeat_front_addr":"172.21.15.12:6829/2509311840","state":["exists","up"]},{"osd":13,"uuid":"37eb42c1-f29e-49f5-b8f6-035178038e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6816","nonce":3716239663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6819","nonce":3716239663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6823","nonce":3716239663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6821","nonce":3716239663}]},"public_addr":"172.21.15.113:6816/3716239663","cluster_addr":"172.21.15.113:6819/3716239663","heartbeat_back_addr":"172.21.15.113:6823/3716239663","heartbeat_front_addr":"172.21.15.113:6821/3716239663","state":["exists","up"]},{"osd":14,"uuid":"b2267367-a857-4a5c-a8d1-6a9b23c748f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6825","nonce":1454956132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6827","nonce":1454956132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6831","nonce":1454956132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6829","nonce":1454956132}]},"public_addr":"172.21.15.137:6825/1454956132","cluster_addr":"172.21.15.137:6827/1454956132","heartbeat_back_addr":"172.21.15.137:6831/1454956132","heartbeat_front_addr":"172.21.15.137:6829/1454956132","state":["exists","up"]},{"osd":15,"uuid":"9933b5aa-e091-43b1-b788-5b79695bab11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6804","nonce":314165740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6807","nonce":314165740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6814","nonce":314165740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6809","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6811","nonce":314165740}]},"public_addr":"172.21.15.178:6804/314165740","cluster_addr":"172.21.15.178:6807/314165740","heartbeat_back_addr":"172.21.15.178:6814/314165740","heartbeat_front_addr":"172.21.15.178:6811/314165740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:30.569382+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:09.218081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:08.352200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:13.912651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:36.408970+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:33.547355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:28.141489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:25.153786+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:26.506163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:10.178898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:20.633419+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:32.266862+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:19.550387+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:10.407663+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:32.078820+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:14.071867+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":20,"length":1},{"begin":22,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:34:04.426 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T04:34:04.426 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T04:34:04.426 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T04:34:04.427 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T04:34:04.427 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T04:34:04.428 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T04:34:04.428 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T04:34:04.428 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T04:34:04.429 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-04-26T04:34:04.429 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-04-26T04:34:04.430 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-04-26T04:34:04.431 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-04-26T04:34:04.431 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-04-26T04:34:04.432 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-04-26T04:34:04.432 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-04-26T04:34:04.433 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-04-26T04:34:04.849 INFO:teuthology.orchestra.run.smithi012.stdout:90194313243 2024-04-26T04:34:04.850 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T04:34:04.885 INFO:teuthology.orchestra.run.smithi012.stdout:98784247836 2024-04-26T04:34:04.886 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T04:34:04.899 INFO:teuthology.orchestra.run.smithi012.stdout:90194313243 2024-04-26T04:34:04.899 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T04:34:04.912 INFO:teuthology.orchestra.run.smithi012.stdout:98784247835 2024-04-26T04:34:04.912 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-04-26T04:34:04.929 INFO:teuthology.orchestra.run.smithi012.stdout:98784247834 2024-04-26T04:34:04.929 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T04:34:04.935 INFO:teuthology.orchestra.run.smithi012.stdout:98784247835 2024-04-26T04:34:04.936 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T04:34:04.956 INFO:teuthology.orchestra.run.smithi012.stdout:94489280539 2024-04-26T04:34:04.956 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T04:34:04.957 INFO:teuthology.orchestra.run.smithi012.stdout:98784247836 2024-04-26T04:34:04.958 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T04:34:04.976 INFO:teuthology.orchestra.run.smithi012.stdout:90194313243 2024-04-26T04:34:04.976 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-04-26T04:34:04.986 INFO:teuthology.orchestra.run.smithi012.stdout:90194313243 2024-04-26T04:34:04.987 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-04-26T04:34:05.000 INFO:teuthology.orchestra.run.smithi012.stdout:90194313243 2024-04-26T04:34:05.001 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-04-26T04:34:05.006 INFO:teuthology.orchestra.run.smithi012.stdout:94489280539 2024-04-26T04:34:05.006 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-04-26T04:34:05.011 INFO:teuthology.orchestra.run.smithi012.stdout:94489280538 2024-04-26T04:34:05.011 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T04:34:05.054 INFO:teuthology.orchestra.run.smithi012.stdout:98784247835 2024-04-26T04:34:05.054 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-04-26T04:34:05.071 INFO:teuthology.orchestra.run.smithi012.stdout:98784247836 2024-04-26T04:34:05.072 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-04-26T04:34:05.087 INFO:teuthology.orchestra.run.smithi012.stdout:98784247836 2024-04-26T04:34:05.088 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-04-26T04:34:05.633 INFO:teuthology.orchestra.run.smithi012.stdout:98784247835 2024-04-26T04:34:05.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247836 got 98784247835 for osd.5 2024-04-26T04:34:05.867 INFO:teuthology.orchestra.run.smithi012.stdout:90194313243 2024-04-26T04:34:05.903 INFO:teuthology.orchestra.run.smithi012.stdout:90194313243 2024-04-26T04:34:05.917 INFO:teuthology.orchestra.run.smithi012.stdout:90194313243 2024-04-26T04:34:05.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313243 got 90194313243 for osd.0 2024-04-26T04:34:05.919 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:05.923 INFO:teuthology.orchestra.run.smithi012.stdout:98784247835 2024-04-26T04:34:05.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313243 got 90194313243 for osd.10 2024-04-26T04:34:05.959 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:05.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313243 got 90194313243 for osd.4 2024-04-26T04:34:05.959 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:05.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247835 got 98784247835 for osd.1 2024-04-26T04:34:05.979 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:05.980 INFO:teuthology.orchestra.run.smithi012.stdout:94489280539 2024-04-26T04:34:05.983 INFO:teuthology.orchestra.run.smithi012.stdout:98784247836 2024-04-26T04:34:06.010 INFO:teuthology.orchestra.run.smithi012.stdout:90194313243 2024-04-26T04:34:06.013 INFO:teuthology.orchestra.run.smithi012.stdout:98784247835 2024-04-26T04:34:06.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280538 got 94489280539 for osd.3 2024-04-26T04:34:06.013 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:06.031 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247836 got 98784247836 for osd.2 2024-04-26T04:34:06.031 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:06.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313243 got 90194313243 for osd.8 2024-04-26T04:34:06.044 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:06.060 INFO:teuthology.orchestra.run.smithi012.stdout:98784247835 2024-04-26T04:34:06.060 INFO:teuthology.orchestra.run.smithi012.stdout:98784247836 2024-04-26T04:34:06.062 INFO:teuthology.orchestra.run.smithi012.stdout:98784247834 2024-04-26T04:34:06.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247835 got 98784247835 for osd.9 2024-04-26T04:34:06.066 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:06.072 INFO:teuthology.orchestra.run.smithi012.stdout:94489280539 2024-04-26T04:34:06.084 INFO:teuthology.orchestra.run.smithi012.stdout:90194313243 2024-04-26T04:34:06.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247835 got 98784247835 for osd.13 2024-04-26T04:34:06.096 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:06.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247834 got 98784247834 for osd.6 2024-04-26T04:34:06.099 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:06.100 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247836 got 98784247836 for osd.14 2024-04-26T04:34:06.100 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:06.102 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280539 got 94489280539 for osd.7 2024-04-26T04:34:06.102 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:06.104 INFO:teuthology.orchestra.run.smithi012.stdout:94489280539 2024-04-26T04:34:06.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313243 got 90194313243 for osd.15 2024-04-26T04:34:06.108 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:06.128 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280539 got 94489280539 for osd.11 2024-04-26T04:34:06.129 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:06.141 INFO:teuthology.orchestra.run.smithi012.stdout:98784247836 2024-04-26T04:34:06.153 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247836 got 98784247836 for osd.12 2024-04-26T04:34:06.153 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:06.690 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T04:34:06.999 INFO:teuthology.orchestra.run.smithi012.stdout:98784247836 2024-04-26T04:34:07.017 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247836 got 98784247836 for osd.5 2024-04-26T04:34:07.017 DEBUG:teuthology.parallel:result is None 2024-04-26T04:34:07.017 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-26T04:34:07.018 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:34:07.297 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:34:07.298 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:34:07.314 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-04-26T04:34:05.360260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":534240,"kb_used_data":31168,"kb_used_omap":24,"kb_used_meta":502823,"kb_avail":1509415200,"statfs":{"total":1546188226560,"available":1545641164800,"internally_reserved":0,"allocated":31916032,"data_stored":19064526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":514890831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.646762"},"pg_stats":[{"pgid":"1.0","version":"32'157","reported_seq":324,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.623586+0000","last_change":"2024-04-26T04:33:48.897362+0000","last_active":"2024-04-26T04:34:04.623586+0000","last_peered":"2024-04-26T04:34:04.623586+0000","last_clean":"2024-04-26T04:34:04.623586+0000","last_became_active":"2024-04-26T04:33:48.796317+0000","last_became_peered":"2024-04-26T04:33:48.796317+0000","last_unstale":"2024-04-26T04:34:04.623586+0000","last_undegraded":"2024-04-26T04:34:04.623586+0000","last_fullsized":"2024-04-26T04:34:04.623586+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'157","last_scrub_stamp":"2024-04-26T04:33:27.245891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:33:27.245891+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.639961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":217,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:03.738447+0000","last_change":"2024-04-26T04:33:19.895062+0000","last_active":"2024-04-26T04:34:03.738447+0000","last_peered":"2024-04-26T04:34:03.738447+0000","last_clean":"2024-04-26T04:34:03.738447+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:34:03.738447+0000","last_undegraded":"2024-04-26T04:34:03.738447+0000","last_fullsized":"2024-04-26T04:34:03.738447+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:19.894942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:19.894942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.035115+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":216,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.755899+0000","last_change":"2024-04-26T04:33:19.350938+0000","last_active":"2024-04-26T04:34:04.755899+0000","last_peered":"2024-04-26T04:34:04.755899+0000","last_clean":"2024-04-26T04:34:04.755899+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:34:04.755899+0000","last_undegraded":"2024-04-26T04:34:04.755899+0000","last_fullsized":"2024-04-26T04:34:04.755899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:19.350538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:19.350538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:36.961751+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.623608+0000","last_change":"2024-04-26T04:33:48.692168+0000","last_active":"2024-04-26T04:34:04.623608+0000","last_peered":"2024-04-26T04:34:04.623608+0000","last_clean":"2024-04-26T04:34:04.623608+0000","last_became_active":"2024-04-26T04:33:48.691427+0000","last_became_peered":"2024-04-26T04:33:48.691427+0000","last_unstale":"2024-04-26T04:34:04.623608+0000","last_undegraded":"2024-04-26T04:34:04.623608+0000","last_fullsized":"2024-04-26T04:34:04.623608+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:38.078086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:38.078086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:01.682032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":215,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.755927+0000","last_change":"2024-04-26T04:33:40.349250+0000","last_active":"2024-04-26T04:34:04.755927+0000","last_peered":"2024-04-26T04:34:04.755927+0000","last_clean":"2024-04-26T04:34:04.755927+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:34:04.755927+0000","last_undegraded":"2024-04-26T04:34:04.755927+0000","last_fullsized":"2024-04-26T04:34:04.755927+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:40.349024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:40.349024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:46.169662+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.621798+0000","last_change":"2024-04-26T04:33:28.610718+0000","last_active":"2024-04-26T04:34:04.621798+0000","last_peered":"2024-04-26T04:34:04.621798+0000","last_clean":"2024-04-26T04:34:04.621798+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:34:04.621798+0000","last_undegraded":"2024-04-26T04:34:04.621798+0000","last_fullsized":"2024-04-26T04:34:04.621798+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:28.610547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:28.610547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:42.699414+0000","scrub_duration":404,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":218,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.626099+0000","last_change":"2024-04-26T04:33:42.761436+0000","last_active":"2024-04-26T04:34:04.626099+0000","last_peered":"2024-04-26T04:34:04.626099+0000","last_clean":"2024-04-26T04:34:04.626099+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:34:04.626099+0000","last_undegraded":"2024-04-26T04:34:04.626099+0000","last_fullsized":"2024-04-26T04:34:04.626099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:42.761200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:42.761200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:53.569844+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.322114+0000","last_change":"2024-04-26T04:33:47.900589+0000","last_active":"2024-04-26T04:34:04.322114+0000","last_peered":"2024-04-26T04:34:04.322114+0000","last_clean":"2024-04-26T04:34:04.322114+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:34:04.322114+0000","last_undegraded":"2024-04-26T04:34:04.322114+0000","last_fullsized":"2024-04-26T04:34:04.322114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:47.900340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:47.900340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:54.638126+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-26T04:34:03.285713+0000","last_change":"2024-04-26T04:33:30.017066+0000","last_active":"2024-04-26T04:34:03.285713+0000","last_peered":"2024-04-26T04:34:03.285713+0000","last_clean":"2024-04-26T04:34:03.285713+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:34:03.285713+0000","last_undegraded":"2024-04-26T04:34:03.285713+0000","last_fullsized":"2024-04-26T04:34:03.285713+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:30.016804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:30.016804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:46.415763+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247835,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33072,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247835,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2469888,"data_stored":1659746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313243,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37220,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334620,"statfs":{"total":96636764160,"available":96598650880,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313243,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313243,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28896,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342944,"statfs":{"total":96636764160,"available":96607174656,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,3,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313243,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247836,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2469888,"data_stored":1659746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280539,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28896,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342944,"statfs":{"total":96636764160,"available":96607174656,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37220,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334620,"statfs":{"total":96636764160,"available":96598650880,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247835,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247836,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247834,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37232,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247836,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33060,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:34:07.315 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:34:07.599 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:34:07.600 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:34:07.617 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-04-26T04:34:07.360913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":534260,"kb_used_data":31184,"kb_used_omap":24,"kb_used_meta":502823,"kb_avail":1509415180,"statfs":{"total":1546188226560,"available":1545641144320,"internally_reserved":0,"allocated":31932416,"data_stored":19074128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":514890831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001833"},"pg_stats":[{"pgid":"1.0","version":"32'157","reported_seq":324,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.623586+0000","last_change":"2024-04-26T04:33:48.897362+0000","last_active":"2024-04-26T04:34:04.623586+0000","last_peered":"2024-04-26T04:34:04.623586+0000","last_clean":"2024-04-26T04:34:04.623586+0000","last_became_active":"2024-04-26T04:33:48.796317+0000","last_became_peered":"2024-04-26T04:33:48.796317+0000","last_unstale":"2024-04-26T04:34:04.623586+0000","last_undegraded":"2024-04-26T04:34:04.623586+0000","last_fullsized":"2024-04-26T04:34:04.623586+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'157","last_scrub_stamp":"2024-04-26T04:33:27.245891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:33:27.245891+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.639961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":217,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:03.738447+0000","last_change":"2024-04-26T04:33:19.895062+0000","last_active":"2024-04-26T04:34:03.738447+0000","last_peered":"2024-04-26T04:34:03.738447+0000","last_clean":"2024-04-26T04:34:03.738447+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:34:03.738447+0000","last_undegraded":"2024-04-26T04:34:03.738447+0000","last_fullsized":"2024-04-26T04:34:03.738447+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:19.894942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:19.894942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.035115+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":216,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.755899+0000","last_change":"2024-04-26T04:33:19.350938+0000","last_active":"2024-04-26T04:34:04.755899+0000","last_peered":"2024-04-26T04:34:04.755899+0000","last_clean":"2024-04-26T04:34:04.755899+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:34:04.755899+0000","last_undegraded":"2024-04-26T04:34:04.755899+0000","last_fullsized":"2024-04-26T04:34:04.755899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:19.350538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:19.350538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:36.961751+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.623608+0000","last_change":"2024-04-26T04:33:48.692168+0000","last_active":"2024-04-26T04:34:04.623608+0000","last_peered":"2024-04-26T04:34:04.623608+0000","last_clean":"2024-04-26T04:34:04.623608+0000","last_became_active":"2024-04-26T04:33:48.691427+0000","last_became_peered":"2024-04-26T04:33:48.691427+0000","last_unstale":"2024-04-26T04:34:04.623608+0000","last_undegraded":"2024-04-26T04:34:04.623608+0000","last_fullsized":"2024-04-26T04:34:04.623608+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:38.078086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:38.078086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:01.682032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":215,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.755927+0000","last_change":"2024-04-26T04:33:40.349250+0000","last_active":"2024-04-26T04:34:04.755927+0000","last_peered":"2024-04-26T04:34:04.755927+0000","last_clean":"2024-04-26T04:34:04.755927+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:34:04.755927+0000","last_undegraded":"2024-04-26T04:34:04.755927+0000","last_fullsized":"2024-04-26T04:34:04.755927+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:40.349024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:40.349024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:46.169662+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.621798+0000","last_change":"2024-04-26T04:33:28.610718+0000","last_active":"2024-04-26T04:34:04.621798+0000","last_peered":"2024-04-26T04:34:04.621798+0000","last_clean":"2024-04-26T04:34:04.621798+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:34:04.621798+0000","last_undegraded":"2024-04-26T04:34:04.621798+0000","last_fullsized":"2024-04-26T04:34:04.621798+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:28.610547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:28.610547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:42.699414+0000","scrub_duration":404,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":218,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.626099+0000","last_change":"2024-04-26T04:33:42.761436+0000","last_active":"2024-04-26T04:34:04.626099+0000","last_peered":"2024-04-26T04:34:04.626099+0000","last_clean":"2024-04-26T04:34:04.626099+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:34:04.626099+0000","last_undegraded":"2024-04-26T04:34:04.626099+0000","last_fullsized":"2024-04-26T04:34:04.626099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:42.761200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:42.761200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:53.569844+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.322114+0000","last_change":"2024-04-26T04:33:47.900589+0000","last_active":"2024-04-26T04:34:04.322114+0000","last_peered":"2024-04-26T04:34:04.322114+0000","last_clean":"2024-04-26T04:34:04.322114+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:34:04.322114+0000","last_undegraded":"2024-04-26T04:34:04.322114+0000","last_fullsized":"2024-04-26T04:34:04.322114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:47.900340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:47.900340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:54.638126+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-26T04:34:03.285713+0000","last_change":"2024-04-26T04:33:30.017066+0000","last_active":"2024-04-26T04:34:03.285713+0000","last_peered":"2024-04-26T04:34:03.285713+0000","last_clean":"2024-04-26T04:34:03.285713+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:34:03.285713+0000","last_undegraded":"2024-04-26T04:34:03.285713+0000","last_fullsized":"2024-04-26T04:34:03.285713+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:30.016804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:30.016804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:46.415763+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33072,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247837,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247836,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2469888,"data_stored":1659746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313243,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37220,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334620,"statfs":{"total":96636764160,"available":96598650880,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313243,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313243,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28896,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342944,"statfs":{"total":96636764160,"available":96607174656,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,3,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313243,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247837,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2469888,"data_stored":1659746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280539,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28896,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342944,"statfs":{"total":96636764160,"available":96607174656,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37220,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334620,"statfs":{"total":96636764160,"available":96598650880,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247837,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247835,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37232,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247837,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33060,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:34:07.618 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-26T04:34:07.618 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-04-26T04:34:07.923 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T04:34:07.933 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T04:34:07.933 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:34:08.224 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:34:08.225 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:34:08.241 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-04-26T04:34:07.360913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":534260,"kb_used_data":31184,"kb_used_omap":24,"kb_used_meta":502823,"kb_avail":1509415180,"statfs":{"total":1546188226560,"available":1545641144320,"internally_reserved":0,"allocated":31932416,"data_stored":19074128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":514890831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001833"},"pg_stats":[{"pgid":"1.0","version":"32'157","reported_seq":324,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.623586+0000","last_change":"2024-04-26T04:33:48.897362+0000","last_active":"2024-04-26T04:34:04.623586+0000","last_peered":"2024-04-26T04:34:04.623586+0000","last_clean":"2024-04-26T04:34:04.623586+0000","last_became_active":"2024-04-26T04:33:48.796317+0000","last_became_peered":"2024-04-26T04:33:48.796317+0000","last_unstale":"2024-04-26T04:34:04.623586+0000","last_undegraded":"2024-04-26T04:34:04.623586+0000","last_fullsized":"2024-04-26T04:34:04.623586+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'157","last_scrub_stamp":"2024-04-26T04:33:27.245891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:33:27.245891+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.639961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":217,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:03.738447+0000","last_change":"2024-04-26T04:33:19.895062+0000","last_active":"2024-04-26T04:34:03.738447+0000","last_peered":"2024-04-26T04:34:03.738447+0000","last_clean":"2024-04-26T04:34:03.738447+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:34:03.738447+0000","last_undegraded":"2024-04-26T04:34:03.738447+0000","last_fullsized":"2024-04-26T04:34:03.738447+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:19.894942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:19.894942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.035115+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":216,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.755899+0000","last_change":"2024-04-26T04:33:19.350938+0000","last_active":"2024-04-26T04:34:04.755899+0000","last_peered":"2024-04-26T04:34:04.755899+0000","last_clean":"2024-04-26T04:34:04.755899+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:34:04.755899+0000","last_undegraded":"2024-04-26T04:34:04.755899+0000","last_fullsized":"2024-04-26T04:34:04.755899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:19.350538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:19.350538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:36.961751+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.623608+0000","last_change":"2024-04-26T04:33:48.692168+0000","last_active":"2024-04-26T04:34:04.623608+0000","last_peered":"2024-04-26T04:34:04.623608+0000","last_clean":"2024-04-26T04:34:04.623608+0000","last_became_active":"2024-04-26T04:33:48.691427+0000","last_became_peered":"2024-04-26T04:33:48.691427+0000","last_unstale":"2024-04-26T04:34:04.623608+0000","last_undegraded":"2024-04-26T04:34:04.623608+0000","last_fullsized":"2024-04-26T04:34:04.623608+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:38.078086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:38.078086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:01.682032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":215,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.755927+0000","last_change":"2024-04-26T04:33:40.349250+0000","last_active":"2024-04-26T04:34:04.755927+0000","last_peered":"2024-04-26T04:34:04.755927+0000","last_clean":"2024-04-26T04:34:04.755927+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:34:04.755927+0000","last_undegraded":"2024-04-26T04:34:04.755927+0000","last_fullsized":"2024-04-26T04:34:04.755927+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:40.349024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:40.349024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:46.169662+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.621798+0000","last_change":"2024-04-26T04:33:28.610718+0000","last_active":"2024-04-26T04:34:04.621798+0000","last_peered":"2024-04-26T04:34:04.621798+0000","last_clean":"2024-04-26T04:34:04.621798+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:34:04.621798+0000","last_undegraded":"2024-04-26T04:34:04.621798+0000","last_fullsized":"2024-04-26T04:34:04.621798+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:28.610547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:28.610547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:42.699414+0000","scrub_duration":404,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":218,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.626099+0000","last_change":"2024-04-26T04:33:42.761436+0000","last_active":"2024-04-26T04:34:04.626099+0000","last_peered":"2024-04-26T04:34:04.626099+0000","last_clean":"2024-04-26T04:34:04.626099+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:34:04.626099+0000","last_undegraded":"2024-04-26T04:34:04.626099+0000","last_fullsized":"2024-04-26T04:34:04.626099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:42.761200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:42.761200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:53.569844+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.322114+0000","last_change":"2024-04-26T04:33:47.900589+0000","last_active":"2024-04-26T04:34:04.322114+0000","last_peered":"2024-04-26T04:34:04.322114+0000","last_clean":"2024-04-26T04:34:04.322114+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:34:04.322114+0000","last_undegraded":"2024-04-26T04:34:04.322114+0000","last_fullsized":"2024-04-26T04:34:04.322114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:47.900340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:47.900340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:54.638126+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-26T04:34:03.285713+0000","last_change":"2024-04-26T04:33:30.017066+0000","last_active":"2024-04-26T04:34:03.285713+0000","last_peered":"2024-04-26T04:34:03.285713+0000","last_clean":"2024-04-26T04:34:03.285713+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:34:03.285713+0000","last_undegraded":"2024-04-26T04:34:03.285713+0000","last_fullsized":"2024-04-26T04:34:03.285713+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:30.016804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:30.016804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:46.415763+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33072,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247837,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247836,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2469888,"data_stored":1659746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313243,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37220,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334620,"statfs":{"total":96636764160,"available":96598650880,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313243,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313243,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28896,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342944,"statfs":{"total":96636764160,"available":96607174656,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,3,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313243,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247837,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2469888,"data_stored":1659746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280539,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28896,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342944,"statfs":{"total":96636764160,"available":96607174656,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37220,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334620,"statfs":{"total":96636764160,"available":96598650880,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247837,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247835,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37232,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247837,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33060,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:34:08.242 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:34:08.537 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:34:08.537 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:34:08.555 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-04-26T04:34:07.360913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":534260,"kb_used_data":31184,"kb_used_omap":24,"kb_used_meta":502823,"kb_avail":1509415180,"statfs":{"total":1546188226560,"available":1545641144320,"internally_reserved":0,"allocated":31932416,"data_stored":19074128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":514890831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001833"},"pg_stats":[{"pgid":"1.0","version":"32'157","reported_seq":324,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.623586+0000","last_change":"2024-04-26T04:33:48.897362+0000","last_active":"2024-04-26T04:34:04.623586+0000","last_peered":"2024-04-26T04:34:04.623586+0000","last_clean":"2024-04-26T04:34:04.623586+0000","last_became_active":"2024-04-26T04:33:48.796317+0000","last_became_peered":"2024-04-26T04:33:48.796317+0000","last_unstale":"2024-04-26T04:34:04.623586+0000","last_undegraded":"2024-04-26T04:34:04.623586+0000","last_fullsized":"2024-04-26T04:34:04.623586+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'157","last_scrub_stamp":"2024-04-26T04:33:27.245891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:33:27.245891+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.639961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":217,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:03.738447+0000","last_change":"2024-04-26T04:33:19.895062+0000","last_active":"2024-04-26T04:34:03.738447+0000","last_peered":"2024-04-26T04:34:03.738447+0000","last_clean":"2024-04-26T04:34:03.738447+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:34:03.738447+0000","last_undegraded":"2024-04-26T04:34:03.738447+0000","last_fullsized":"2024-04-26T04:34:03.738447+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:19.894942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:19.894942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.035115+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":216,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.755899+0000","last_change":"2024-04-26T04:33:19.350938+0000","last_active":"2024-04-26T04:34:04.755899+0000","last_peered":"2024-04-26T04:34:04.755899+0000","last_clean":"2024-04-26T04:34:04.755899+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:34:04.755899+0000","last_undegraded":"2024-04-26T04:34:04.755899+0000","last_fullsized":"2024-04-26T04:34:04.755899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:19.350538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:19.350538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:36.961751+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.623608+0000","last_change":"2024-04-26T04:33:48.692168+0000","last_active":"2024-04-26T04:34:04.623608+0000","last_peered":"2024-04-26T04:34:04.623608+0000","last_clean":"2024-04-26T04:34:04.623608+0000","last_became_active":"2024-04-26T04:33:48.691427+0000","last_became_peered":"2024-04-26T04:33:48.691427+0000","last_unstale":"2024-04-26T04:34:04.623608+0000","last_undegraded":"2024-04-26T04:34:04.623608+0000","last_fullsized":"2024-04-26T04:34:04.623608+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:38.078086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:38.078086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:01.682032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":215,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.755927+0000","last_change":"2024-04-26T04:33:40.349250+0000","last_active":"2024-04-26T04:34:04.755927+0000","last_peered":"2024-04-26T04:34:04.755927+0000","last_clean":"2024-04-26T04:34:04.755927+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:34:04.755927+0000","last_undegraded":"2024-04-26T04:34:04.755927+0000","last_fullsized":"2024-04-26T04:34:04.755927+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:40.349024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:40.349024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:46.169662+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.621798+0000","last_change":"2024-04-26T04:33:28.610718+0000","last_active":"2024-04-26T04:34:04.621798+0000","last_peered":"2024-04-26T04:34:04.621798+0000","last_clean":"2024-04-26T04:34:04.621798+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:34:04.621798+0000","last_undegraded":"2024-04-26T04:34:04.621798+0000","last_fullsized":"2024-04-26T04:34:04.621798+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:28.610547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:28.610547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:42.699414+0000","scrub_duration":404,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":218,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.626099+0000","last_change":"2024-04-26T04:33:42.761436+0000","last_active":"2024-04-26T04:34:04.626099+0000","last_peered":"2024-04-26T04:34:04.626099+0000","last_clean":"2024-04-26T04:34:04.626099+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:34:04.626099+0000","last_undegraded":"2024-04-26T04:34:04.626099+0000","last_fullsized":"2024-04-26T04:34:04.626099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:42.761200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:42.761200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:53.569844+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.322114+0000","last_change":"2024-04-26T04:33:47.900589+0000","last_active":"2024-04-26T04:34:04.322114+0000","last_peered":"2024-04-26T04:34:04.322114+0000","last_clean":"2024-04-26T04:34:04.322114+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:34:04.322114+0000","last_undegraded":"2024-04-26T04:34:04.322114+0000","last_fullsized":"2024-04-26T04:34:04.322114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:47.900340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:47.900340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:54.638126+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-26T04:34:03.285713+0000","last_change":"2024-04-26T04:33:30.017066+0000","last_active":"2024-04-26T04:34:03.285713+0000","last_peered":"2024-04-26T04:34:03.285713+0000","last_clean":"2024-04-26T04:34:03.285713+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:34:03.285713+0000","last_undegraded":"2024-04-26T04:34:03.285713+0000","last_fullsized":"2024-04-26T04:34:03.285713+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:30.016804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:30.016804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:46.415763+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33072,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247837,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247836,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2469888,"data_stored":1659746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313243,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37220,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334620,"statfs":{"total":96636764160,"available":96598650880,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313243,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313243,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28896,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342944,"statfs":{"total":96636764160,"available":96607174656,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,3,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313243,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247837,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2469888,"data_stored":1659746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280539,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28896,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342944,"statfs":{"total":96636764160,"available":96607174656,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37220,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334620,"statfs":{"total":96636764160,"available":96598650880,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247837,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247835,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37232,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247837,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33060,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:34:08.557 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T04:34:08.557 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:34:08.852 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:34:08.852 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:34:08.869 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-04-26T04:34:07.360913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":534260,"kb_used_data":31184,"kb_used_omap":24,"kb_used_meta":502823,"kb_avail":1509415180,"statfs":{"total":1546188226560,"available":1545641144320,"internally_reserved":0,"allocated":31932416,"data_stored":19074128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":514890831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001833"},"pg_stats":[{"pgid":"1.0","version":"32'157","reported_seq":324,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.623586+0000","last_change":"2024-04-26T04:33:48.897362+0000","last_active":"2024-04-26T04:34:04.623586+0000","last_peered":"2024-04-26T04:34:04.623586+0000","last_clean":"2024-04-26T04:34:04.623586+0000","last_became_active":"2024-04-26T04:33:48.796317+0000","last_became_peered":"2024-04-26T04:33:48.796317+0000","last_unstale":"2024-04-26T04:34:04.623586+0000","last_undegraded":"2024-04-26T04:34:04.623586+0000","last_fullsized":"2024-04-26T04:34:04.623586+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'157","last_scrub_stamp":"2024-04-26T04:33:27.245891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:33:27.245891+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.639961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":217,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:03.738447+0000","last_change":"2024-04-26T04:33:19.895062+0000","last_active":"2024-04-26T04:34:03.738447+0000","last_peered":"2024-04-26T04:34:03.738447+0000","last_clean":"2024-04-26T04:34:03.738447+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:34:03.738447+0000","last_undegraded":"2024-04-26T04:34:03.738447+0000","last_fullsized":"2024-04-26T04:34:03.738447+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:19.894942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:19.894942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:34.035115+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":216,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.755899+0000","last_change":"2024-04-26T04:33:19.350938+0000","last_active":"2024-04-26T04:34:04.755899+0000","last_peered":"2024-04-26T04:34:04.755899+0000","last_clean":"2024-04-26T04:34:04.755899+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:34:04.755899+0000","last_undegraded":"2024-04-26T04:34:04.755899+0000","last_fullsized":"2024-04-26T04:34:04.755899+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:19.350538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:19.350538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:36.961751+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":112,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.623608+0000","last_change":"2024-04-26T04:33:48.692168+0000","last_active":"2024-04-26T04:34:04.623608+0000","last_peered":"2024-04-26T04:34:04.623608+0000","last_clean":"2024-04-26T04:34:04.623608+0000","last_became_active":"2024-04-26T04:33:48.691427+0000","last_became_peered":"2024-04-26T04:33:48.691427+0000","last_unstale":"2024-04-26T04:34:04.623608+0000","last_undegraded":"2024-04-26T04:34:04.623608+0000","last_fullsized":"2024-04-26T04:34:04.623608+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:38.078086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:38.078086+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:01.682032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":215,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.755927+0000","last_change":"2024-04-26T04:33:40.349250+0000","last_active":"2024-04-26T04:34:04.755927+0000","last_peered":"2024-04-26T04:34:04.755927+0000","last_clean":"2024-04-26T04:34:04.755927+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:34:04.755927+0000","last_undegraded":"2024-04-26T04:34:04.755927+0000","last_fullsized":"2024-04-26T04:34:04.755927+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:40.349024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:40.349024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:46.169662+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.621798+0000","last_change":"2024-04-26T04:33:28.610718+0000","last_active":"2024-04-26T04:34:04.621798+0000","last_peered":"2024-04-26T04:34:04.621798+0000","last_clean":"2024-04-26T04:34:04.621798+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:34:04.621798+0000","last_undegraded":"2024-04-26T04:34:04.621798+0000","last_fullsized":"2024-04-26T04:34:04.621798+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:28.610547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:28.610547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:42.699414+0000","scrub_duration":404,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":218,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.626099+0000","last_change":"2024-04-26T04:33:42.761436+0000","last_active":"2024-04-26T04:34:04.626099+0000","last_peered":"2024-04-26T04:34:04.626099+0000","last_clean":"2024-04-26T04:34:04.626099+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:34:04.626099+0000","last_undegraded":"2024-04-26T04:34:04.626099+0000","last_fullsized":"2024-04-26T04:34:04.626099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:42.761200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:42.761200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:53.569844+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":214,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.322114+0000","last_change":"2024-04-26T04:33:47.900589+0000","last_active":"2024-04-26T04:34:04.322114+0000","last_peered":"2024-04-26T04:34:04.322114+0000","last_clean":"2024-04-26T04:34:04.322114+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:34:04.322114+0000","last_undegraded":"2024-04-26T04:34:04.322114+0000","last_fullsized":"2024-04-26T04:34:04.322114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:47.900340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:47.900340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:54.638126+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":213,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-26T04:34:03.285713+0000","last_change":"2024-04-26T04:33:30.017066+0000","last_active":"2024-04-26T04:34:03.285713+0000","last_peered":"2024-04-26T04:34:03.285713+0000","last_clean":"2024-04-26T04:34:03.285713+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:34:03.285713+0000","last_undegraded":"2024-04-26T04:34:03.285713+0000","last_fullsized":"2024-04-26T04:34:03.285713+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:30.016804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:30.016804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:46.415763+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33072,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247837,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247836,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2469888,"data_stored":1659746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313243,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37220,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334620,"statfs":{"total":96636764160,"available":96598650880,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313243,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313243,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313243,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28896,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342944,"statfs":{"total":96636764160,"available":96607174656,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,3,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313243,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247837,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2469888,"data_stored":1659746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280539,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28896,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342944,"statfs":{"total":96636764160,"available":96607174656,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37220,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334620,"statfs":{"total":96636764160,"available":96598650880,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247836,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247837,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247835,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37232,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247837,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33060,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1921024,"data_stored":1121216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:34:08.870 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T04:34:09.169 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:34:09.169 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":128,"fsid":"12c27fbe-0161-48d0-88ec-42a226685437","created":"2024-04-26T04:31:56.451062+0000","modified":"2024-04-26T04:34:03.731364+0000","last_up_change":"2024-04-26T04:32:09.452464+0000","last_in_change":"2024-04-26T04:33:31.417237+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:32:07.426875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-26T04:32:11.318957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"55a4f254-2c2a-4007-9c52-b89df6fffb69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6816","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6817","nonce":1754063127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6818","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6819","nonce":1754063127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6822","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6823","nonce":1754063127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6820","nonce":1754063127},{"type":"v1","addr":"172.21.15.12:6821","nonce":1754063127}]},"public_addr":"172.21.15.12:6817/1754063127","cluster_addr":"172.21.15.12:6819/1754063127","heartbeat_back_addr":"172.21.15.12:6823/1754063127","heartbeat_front_addr":"172.21.15.12:6821/1754063127","state":["exists","up"]},{"osd":1,"uuid":"32db4f97-c8ac-481c-9e80-18fc91004298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6809","nonce":3264326676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6811","nonce":3264326676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6817","nonce":3264326676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3264326676},{"type":"v1","addr":"172.21.15.113:6813","nonce":3264326676}]},"public_addr":"172.21.15.113:6809/3264326676","cluster_addr":"172.21.15.113:6811/3264326676","heartbeat_back_addr":"172.21.15.113:6817/3264326676","heartbeat_front_addr":"172.21.15.113:6813/3264326676","state":["exists","up"]},{"osd":2,"uuid":"2cd26466-5d59-4599-ba3f-a78b0e3b681f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6802","nonce":1454475448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6806","nonce":1454475448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6814","nonce":1454475448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":1454475448},{"type":"v1","addr":"172.21.15.137:6810","nonce":1454475448}]},"public_addr":"172.21.15.137:6802/1454475448","cluster_addr":"172.21.15.137:6806/1454475448","heartbeat_back_addr":"172.21.15.137:6814/1454475448","heartbeat_front_addr":"172.21.15.137:6810/1454475448","state":["exists","up"]},{"osd":3,"uuid":"0945017d-9525-49d6-aa67-6635acd94c3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6803","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6805","nonce":2898698985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6810","nonce":2898698985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6817","nonce":2898698985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6813","nonce":2898698985},{"type":"v1","addr":"172.21.15.178:6815","nonce":2898698985}]},"public_addr":"172.21.15.178:6805/2898698985","cluster_addr":"172.21.15.178:6810/2898698985","heartbeat_back_addr":"172.21.15.178:6817/2898698985","heartbeat_front_addr":"172.21.15.178:6815/2898698985","state":["exists","up"]},{"osd":4,"uuid":"38dc68bf-0a89-4900-9d35-24c70368447a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6808","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6809","nonce":3209840823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6810","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6811","nonce":3209840823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6814","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6815","nonce":3209840823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6812","nonce":3209840823},{"type":"v1","addr":"172.21.15.12:6813","nonce":3209840823}]},"public_addr":"172.21.15.12:6809/3209840823","cluster_addr":"172.21.15.12:6811/3209840823","heartbeat_back_addr":"172.21.15.12:6815/3209840823","heartbeat_front_addr":"172.21.15.12:6813/3209840823","state":["exists","up"]},{"osd":5,"uuid":"e91cc8a1-57e1-4760-a329-8a60aa752a95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6801","nonce":1475637773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6803","nonce":1475637773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6807","nonce":1475637773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":1475637773},{"type":"v1","addr":"172.21.15.113:6805","nonce":1475637773}]},"public_addr":"172.21.15.113:6801/1475637773","cluster_addr":"172.21.15.113:6803/1475637773","heartbeat_back_addr":"172.21.15.113:6807/1475637773","heartbeat_front_addr":"172.21.15.113:6805/1475637773","state":["exists","up"]},{"osd":6,"uuid":"a1322130-3391-432d-a1ee-f83297885029","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6817","nonce":771338783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6819","nonce":771338783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6823","nonce":771338783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":771338783},{"type":"v1","addr":"172.21.15.137:6821","nonce":771338783}]},"public_addr":"172.21.15.137:6817/771338783","cluster_addr":"172.21.15.137:6819/771338783","heartbeat_back_addr":"172.21.15.137:6823/771338783","heartbeat_front_addr":"172.21.15.137:6821/771338783","state":["exists","up"]},{"osd":7,"uuid":"02ee4ca2-9d48-4aff-9641-1823760c7a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6827","nonce":1017152077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6829","nonce":1017152077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6833","nonce":1017152077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":1017152077},{"type":"v1","addr":"172.21.15.178:6831","nonce":1017152077}]},"public_addr":"172.21.15.178:6827/1017152077","cluster_addr":"172.21.15.178:6829/1017152077","heartbeat_back_addr":"172.21.15.178:6833/1017152077","heartbeat_front_addr":"172.21.15.178:6831/1017152077","state":["exists","up"]},{"osd":8,"uuid":"ea3d5f62-3d68-4824-8556-da8db36553fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6801","nonce":2153129138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6803","nonce":2153129138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6807","nonce":2153129138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2153129138},{"type":"v1","addr":"172.21.15.12:6805","nonce":2153129138}]},"public_addr":"172.21.15.12:6801/2153129138","cluster_addr":"172.21.15.12:6803/2153129138","heartbeat_back_addr":"172.21.15.12:6807/2153129138","heartbeat_front_addr":"172.21.15.12:6805/2153129138","state":["exists","up"]},{"osd":9,"uuid":"e653d5bd-6925-4e6f-a04a-cb7c7bcc5f70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6825","nonce":359931551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6827","nonce":359931551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6831","nonce":359931551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":359931551},{"type":"v1","addr":"172.21.15.113:6829","nonce":359931551}]},"public_addr":"172.21.15.113:6825/359931551","cluster_addr":"172.21.15.113:6827/359931551","heartbeat_back_addr":"172.21.15.113:6831/359931551","heartbeat_front_addr":"172.21.15.113:6829/359931551","state":["exists","up"]},{"osd":10,"uuid":"8c4f8df5-465a-486e-9413-1de14e33c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6801","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6803","nonce":1533480861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6805","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6807","nonce":1533480861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6813","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6815","nonce":1533480861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6809","nonce":1533480861},{"type":"v1","addr":"172.21.15.137:6811","nonce":1533480861}]},"public_addr":"172.21.15.137:6803/1533480861","cluster_addr":"172.21.15.137:6807/1533480861","heartbeat_back_addr":"172.21.15.137:6815/1533480861","heartbeat_front_addr":"172.21.15.137:6811/1533480861","state":["exists","up"]},{"osd":11,"uuid":"6fe54c2d-0b95-4235-8c22-51230cfaa591","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6819","nonce":1683682957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6821","nonce":1683682957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6825","nonce":1683682957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":1683682957},{"type":"v1","addr":"172.21.15.178:6823","nonce":1683682957}]},"public_addr":"172.21.15.178:6819/1683682957","cluster_addr":"172.21.15.178:6821/1683682957","heartbeat_back_addr":"172.21.15.178:6825/1683682957","heartbeat_front_addr":"172.21.15.178:6823/1683682957","state":["exists","up"]},{"osd":12,"uuid":"5ddd0f05-fb78-43bb-95ec-3fc7985603b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6824","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6825","nonce":2509311840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6826","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6827","nonce":2509311840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6830","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6831","nonce":2509311840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6828","nonce":2509311840},{"type":"v1","addr":"172.21.15.12:6829","nonce":2509311840}]},"public_addr":"172.21.15.12:6825/2509311840","cluster_addr":"172.21.15.12:6827/2509311840","heartbeat_back_addr":"172.21.15.12:6831/2509311840","heartbeat_front_addr":"172.21.15.12:6829/2509311840","state":["exists","up"]},{"osd":13,"uuid":"37eb42c1-f29e-49f5-b8f6-035178038e5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6815","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6816","nonce":3716239663}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6819","nonce":3716239663}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6823","nonce":3716239663}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3716239663},{"type":"v1","addr":"172.21.15.113:6821","nonce":3716239663}]},"public_addr":"172.21.15.113:6816/3716239663","cluster_addr":"172.21.15.113:6819/3716239663","heartbeat_back_addr":"172.21.15.113:6823/3716239663","heartbeat_front_addr":"172.21.15.113:6821/3716239663","state":["exists","up"]},{"osd":14,"uuid":"b2267367-a857-4a5c-a8d1-6a9b23c748f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6825","nonce":1454956132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6827","nonce":1454956132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6831","nonce":1454956132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":1454956132},{"type":"v1","addr":"172.21.15.137:6829","nonce":1454956132}]},"public_addr":"172.21.15.137:6825/1454956132","cluster_addr":"172.21.15.137:6827/1454956132","heartbeat_back_addr":"172.21.15.137:6831/1454956132","heartbeat_front_addr":"172.21.15.137:6829/1454956132","state":["exists","up"]},{"osd":15,"uuid":"9933b5aa-e091-43b1-b788-5b79695bab11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6804","nonce":314165740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6807","nonce":314165740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6814","nonce":314165740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6809","nonce":314165740},{"type":"v1","addr":"172.21.15.178:6811","nonce":314165740}]},"public_addr":"172.21.15.178:6804/314165740","cluster_addr":"172.21.15.178:6807/314165740","heartbeat_back_addr":"172.21.15.178:6814/314165740","heartbeat_front_addr":"172.21.15.178:6811/314165740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:30.569382+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:09.218081+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:08.352200+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:13.912651+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:36.408970+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:33.547355+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:28.141489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:25.153786+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:26.506163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:10.178898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:20.633419+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:32.266862+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:19.550387+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:10.407663+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:32.078820+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:33:14.071867+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":20,"length":1},{"begin":22,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:34:10.189 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T04:34:10.190 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:10.352 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:10.352 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:10.352 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:10.369 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T04:34:10.666 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T04:34:10.686 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T04:34:10.686 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:10.844 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:10.844 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:10.844 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:10.861 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T04:34:11.155 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T04:34:11.174 INFO:tasks.ceph:Scrubbing osd.8 2024-04-26T04:34:11.174 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:11.326 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:11.326 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:11.326 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:11.341 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2024-04-26T04:34:11.631 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 8 to deep-scrub 2024-04-26T04:34:11.649 INFO:tasks.ceph:Scrubbing osd.12 2024-04-26T04:34:11.650 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:11.802 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:11.802 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:11.802 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:11.818 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 12 2024-04-26T04:34:12.109 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 12 to deep-scrub 2024-04-26T04:34:12.128 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T04:34:12.129 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:12.277 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:12.278 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:12.278 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:12.294 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T04:34:12.588 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T04:34:12.607 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T04:34:12.607 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:12.766 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:12.766 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:12.766 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:12.781 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T04:34:13.080 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T04:34:13.098 INFO:tasks.ceph:Scrubbing osd.9 2024-04-26T04:34:13.099 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:13.258 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:13.258 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:13.258 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:13.274 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2024-04-26T04:34:13.569 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 9 to deep-scrub 2024-04-26T04:34:13.591 INFO:tasks.ceph:Scrubbing osd.13 2024-04-26T04:34:13.591 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:13.736 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:13.736 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:13.736 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:13.751 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 13 2024-04-26T04:34:14.030 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 13 to deep-scrub 2024-04-26T04:34:14.047 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T04:34:14.048 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:14.194 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:14.194 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:14.194 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:14.209 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T04:34:14.495 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T04:34:14.513 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T04:34:14.513 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:14.656 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:14.656 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:14.656 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:14.672 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T04:34:14.949 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T04:34:14.966 INFO:tasks.ceph:Scrubbing osd.10 2024-04-26T04:34:14.966 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:15.108 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:15.109 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:15.109 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:15.123 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2024-04-26T04:34:15.418 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 10 to deep-scrub 2024-04-26T04:34:15.435 INFO:tasks.ceph:Scrubbing osd.14 2024-04-26T04:34:15.435 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:15.592 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:15.592 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:15.592 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:15.608 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 14 2024-04-26T04:34:15.888 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 14 to deep-scrub 2024-04-26T04:34:15.907 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T04:34:15.907 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:16.061 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:16.061 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:16.061 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:16.078 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T04:34:16.370 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T04:34:16.391 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T04:34:16.391 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:16.546 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:16.546 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:16.547 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:16.562 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T04:34:16.855 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T04:34:16.875 INFO:tasks.ceph:Scrubbing osd.11 2024-04-26T04:34:16.875 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:17.036 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:17.037 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:17.037 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:17.054 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2024-04-26T04:34:17.342 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 11 to deep-scrub 2024-04-26T04:34:17.360 INFO:tasks.ceph:Scrubbing osd.15 2024-04-26T04:34:17.360 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 config set osd_debug_deep_scrub_sleep 0 2024-04-26T04:34:17.514 INFO:teuthology.orchestra.run.smithi012.stdout:{ 2024-04-26T04:34:17.514 INFO:teuthology.orchestra.run.smithi012.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T04:34:17.514 INFO:teuthology.orchestra.run.smithi012.stdout:} 2024-04-26T04:34:17.529 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 15 2024-04-26T04:34:17.817 INFO:teuthology.orchestra.run.smithi012.stderr:instructed osd(s) 15 to deep-scrub 2024-04-26T04:34:17.835 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:34:18.120 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:34:18.120 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:34:18.137 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-04-26T04:34:17.363988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":534396,"kb_used_data":31312,"kb_used_omap":24,"kb_used_meta":502823,"kb_avail":1509415044,"statfs":{"total":1546188226560,"available":1545641005056,"internally_reserved":0,"allocated":32063488,"data_stored":19150944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":514890831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003727"},"pg_stats":[{"pgid":"1.0","version":"32'157","reported_seq":326,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:10.625184+0000","last_change":"2024-04-26T04:33:48.897362+0000","last_active":"2024-04-26T04:34:10.625184+0000","last_peered":"2024-04-26T04:34:10.625184+0000","last_clean":"2024-04-26T04:34:10.625184+0000","last_became_active":"2024-04-26T04:33:48.796317+0000","last_became_peered":"2024-04-26T04:33:48.796317+0000","last_unstale":"2024-04-26T04:34:10.625184+0000","last_undegraded":"2024-04-26T04:34:10.625184+0000","last_fullsized":"2024-04-26T04:34:10.625184+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'157","last_scrub_stamp":"2024-04-26T04:33:27.245891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:08.365921+0000","last_clean_scrub_stamp":"2024-04-26T04:33:27.245891+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":226,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:14.372413+0000","last_change":"2024-04-26T04:34:14.372465+0000","last_active":"2024-04-26T04:34:14.372413+0000","last_peered":"2024-04-26T04:34:14.372413+0000","last_clean":"2024-04-26T04:34:14.372413+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:34:14.372413+0000","last_undegraded":"2024-04-26T04:34:14.372413+0000","last_fullsized":"2024-04-26T04:34:14.372413+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:34:14.372227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:34:14.372227+0000","last_clean_scrub_stamp":"2024-04-26T04:34:14.372227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:15.338005+0000","scrub_duration":407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":217,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.755975+0000","last_change":"2024-04-26T04:33:19.350938+0000","last_active":"2024-04-26T04:34:04.755975+0000","last_peered":"2024-04-26T04:34:04.755975+0000","last_clean":"2024-04-26T04:34:04.755975+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:34:04.755975+0000","last_undegraded":"2024-04-26T04:34:04.755975+0000","last_fullsized":"2024-04-26T04:34:04.755975+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:19.350538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:19.350538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:36.961751+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":122,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:15.014315+0000","last_change":"2024-04-26T04:34:15.014315+0000","last_active":"2024-04-26T04:34:15.014315+0000","last_peered":"2024-04-26T04:34:15.014315+0000","last_clean":"2024-04-26T04:34:15.014315+0000","last_became_active":"2024-04-26T04:33:48.691427+0000","last_became_peered":"2024-04-26T04:33:48.691427+0000","last_unstale":"2024-04-26T04:34:15.014315+0000","last_undegraded":"2024-04-26T04:34:15.014315+0000","last_fullsized":"2024-04-26T04:34:15.014315+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:34:15.014076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:34:15.014076+0000","last_clean_scrub_stamp":"2024-04-26T04:34:15.014076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:28.053899+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":215,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.755927+0000","last_change":"2024-04-26T04:33:40.349250+0000","last_active":"2024-04-26T04:34:04.755927+0000","last_peered":"2024-04-26T04:34:04.755927+0000","last_clean":"2024-04-26T04:34:04.755927+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:34:04.755927+0000","last_undegraded":"2024-04-26T04:34:04.755927+0000","last_fullsized":"2024-04-26T04:34:04.755927+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:40.349024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:40.349024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:46.169662+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":223,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:12.360226+0000","last_change":"2024-04-26T04:34:12.360251+0000","last_active":"2024-04-26T04:34:12.360226+0000","last_peered":"2024-04-26T04:34:12.360226+0000","last_clean":"2024-04-26T04:34:12.360226+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:34:12.360226+0000","last_undegraded":"2024-04-26T04:34:12.360226+0000","last_fullsized":"2024-04-26T04:34:12.360226+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:34:12.360111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:34:12.360111+0000","last_clean_scrub_stamp":"2024-04-26T04:34:12.360111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:14.687856+0000","scrub_duration":399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":219,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:04.626234+0000","last_change":"2024-04-26T04:33:42.761436+0000","last_active":"2024-04-26T04:34:04.626234+0000","last_peered":"2024-04-26T04:34:04.626234+0000","last_clean":"2024-04-26T04:34:04.626234+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:34:04.626234+0000","last_undegraded":"2024-04-26T04:34:04.626234+0000","last_fullsized":"2024-04-26T04:34:04.626234+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:42.761200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:42.761200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":223,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:12.649344+0000","last_change":"2024-04-26T04:34:12.649437+0000","last_active":"2024-04-26T04:34:12.649344+0000","last_peered":"2024-04-26T04:34:12.649344+0000","last_clean":"2024-04-26T04:34:12.649344+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:34:12.649344+0000","last_undegraded":"2024-04-26T04:34:12.649344+0000","last_fullsized":"2024-04-26T04:34:12.649344+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:34:12.649215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:34:12.649215+0000","last_clean_scrub_stamp":"2024-04-26T04:34:12.649215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:22.878039+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":215,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-26T04:34:05.008016+0000","last_change":"2024-04-26T04:33:30.017066+0000","last_active":"2024-04-26T04:34:05.008016+0000","last_peered":"2024-04-26T04:34:05.008016+0000","last_clean":"2024-04-26T04:34:05.008016+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:34:05.008016+0000","last_undegraded":"2024-04-26T04:34:05.008016+0000","last_fullsized":"2024-04-26T04:34:05.008016+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:33:30.016804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:32:11.466714+0000","last_clean_scrub_stamp":"2024-04-26T04:33:30.016804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:46.415763+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247838,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33072,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247839,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247838,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2469888,"data_stored":1659746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37232,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313245,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313245,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313245,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28916,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342924,"statfs":{"total":96636764160,"available":96607154176,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,3,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313245,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247839,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33596,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2469888,"data_stored":1659746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280541,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28916,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342924,"statfs":{"total":96636764160,"available":96607154176,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280541,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37240,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334600,"statfs":{"total":96636764160,"available":96598630400,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247838,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247839,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33068,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338772,"statfs":{"total":96636764160,"available":96602902528,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247837,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280541,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37232,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334608,"statfs":{"total":96636764160,"available":96598638592,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247839,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1937408,"data_stored":1130818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:34:18.141 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T04:33:27.245891+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=4, tm_min=33, tm_sec=27, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=4, tm_min=34, tm_sec=9, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T04:34:18.141 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-26T04:33:19.350538+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=4, tm_min=33, tm_sec=19, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=4, tm_min=34, tm_sec=9, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T04:34:18.141 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-26T04:33:40.349024+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=4, tm_min=33, tm_sec=40, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=4, tm_min=34, tm_sec=9, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T04:34:18.141 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-26T04:33:42.761200+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=4, tm_min=33, tm_sec=42, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=4, tm_min=34, tm_sec=9, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T04:34:18.141 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-26T04:33:30.016804+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=4, tm_min=33, tm_sec=30, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=4, tm_min=34, tm_sec=9, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T04:34:18.141 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T04:34:38.142 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T04:34:38.455 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-04-26T04:34:38.456 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-04-26T04:34:38.475 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-04-26T04:34:37.369921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":535568,"kb_used_data":32512,"kb_used_omap":24,"kb_used_meta":502823,"kb_avail":1509413872,"statfs":{"total":1546188226560,"available":1545639804928,"internally_reserved":0,"allocated":33292288,"data_stored":19875044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":514890831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.872263"},"pg_stats":[{"pgid":"1.0","version":"128'159","reported_seq":346,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-04-26T04:34:33.971833+0000","last_change":"2024-04-26T04:34:15.930588+0000","last_active":"2024-04-26T04:34:33.971833+0000","last_peered":"2024-04-26T04:34:33.971833+0000","last_clean":"2024-04-26T04:34:33.971833+0000","last_became_active":"2024-04-26T04:33:48.796317+0000","last_became_peered":"2024-04-26T04:33:48.796317+0000","last_unstale":"2024-04-26T04:34:33.971833+0000","last_undegraded":"2024-04-26T04:34:33.971833+0000","last_fullsized":"2024-04-26T04:34:33.971833+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'159","last_scrub_stamp":"2024-04-26T04:34:15.930209+0000","last_deep_scrub":"128'159","last_deep_scrub_stamp":"2024-04-26T04:34:15.930209+0000","last_clean_scrub_stamp":"2024-04-26T04:34:15.930209+0000","objects_scrubbed":2,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:42.764118+0000","scrub_duration":111,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":237,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-04-26T04:34:33.729214+0000","last_change":"2024-04-26T04:34:14.372465+0000","last_active":"2024-04-26T04:34:33.729214+0000","last_peered":"2024-04-26T04:34:33.729214+0000","last_clean":"2024-04-26T04:34:33.729214+0000","last_became_active":"2024-04-26T04:32:12.504116+0000","last_became_peered":"2024-04-26T04:32:12.504116+0000","last_unstale":"2024-04-26T04:34:33.729214+0000","last_undegraded":"2024-04-26T04:34:33.729214+0000","last_fullsized":"2024-04-26T04:34:33.729214+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:34:14.372227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:34:14.372227+0000","last_clean_scrub_stamp":"2024-04-26T04:34:14.372227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:15.338005+0000","scrub_duration":407,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":236,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-04-26T04:34:33.971457+0000","last_change":"2024-04-26T04:34:17.286390+0000","last_active":"2024-04-26T04:34:33.971457+0000","last_peered":"2024-04-26T04:34:33.971457+0000","last_clean":"2024-04-26T04:34:33.971457+0000","last_became_active":"2024-04-26T04:32:12.602820+0000","last_became_peered":"2024-04-26T04:32:12.602820+0000","last_unstale":"2024-04-26T04:34:33.971457+0000","last_undegraded":"2024-04-26T04:34:33.971457+0000","last_fullsized":"2024-04-26T04:34:33.971457+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:34:17.286151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:34:17.286151+0000","last_clean_scrub_stamp":"2024-04-26T04:34:17.286151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:42.727563+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-04-26T04:34:33.971816+0000","last_change":"2024-04-26T04:34:15.014315+0000","last_active":"2024-04-26T04:34:33.971816+0000","last_peered":"2024-04-26T04:34:33.971816+0000","last_clean":"2024-04-26T04:34:33.971816+0000","last_became_active":"2024-04-26T04:33:48.691427+0000","last_became_peered":"2024-04-26T04:33:48.691427+0000","last_unstale":"2024-04-26T04:34:33.971816+0000","last_undegraded":"2024-04-26T04:34:33.971816+0000","last_fullsized":"2024-04-26T04:34:33.971816+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:34:15.014076+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:34:15.014076+0000","last_clean_scrub_stamp":"2024-04-26T04:34:15.014076+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:28.053899+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":235,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-04-26T04:34:33.971455+0000","last_change":"2024-04-26T04:34:18.540627+0000","last_active":"2024-04-26T04:34:33.971455+0000","last_peered":"2024-04-26T04:34:33.971455+0000","last_clean":"2024-04-26T04:34:33.971455+0000","last_became_active":"2024-04-26T04:32:12.797825+0000","last_became_peered":"2024-04-26T04:32:12.797825+0000","last_unstale":"2024-04-26T04:34:33.971455+0000","last_undegraded":"2024-04-26T04:34:33.971455+0000","last_fullsized":"2024-04-26T04:34:33.971455+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:34:18.540392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:34:18.540392+0000","last_clean_scrub_stamp":"2024-04-26T04:34:18.540392+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:34.040428+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":232,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T04:34:29.568094+0000","last_change":"2024-04-26T04:34:12.360251+0000","last_active":"2024-04-26T04:34:29.568094+0000","last_peered":"2024-04-26T04:34:29.568094+0000","last_clean":"2024-04-26T04:34:29.568094+0000","last_became_active":"2024-04-26T04:32:13.008132+0000","last_became_peered":"2024-04-26T04:32:13.008132+0000","last_unstale":"2024-04-26T04:34:29.568094+0000","last_undegraded":"2024-04-26T04:34:29.568094+0000","last_fullsized":"2024-04-26T04:34:29.568094+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:34:12.360111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:34:12.360111+0000","last_clean_scrub_stamp":"2024-04-26T04:34:12.360111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:14.687856+0000","scrub_duration":399,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":238,"reported_epoch":133,"state":"active+clean","last_fresh":"2024-04-26T04:34:33.531200+0000","last_change":"2024-04-26T04:34:15.448141+0000","last_active":"2024-04-26T04:34:33.531200+0000","last_peered":"2024-04-26T04:34:33.531200+0000","last_clean":"2024-04-26T04:34:33.531200+0000","last_became_active":"2024-04-26T04:32:12.503724+0000","last_became_peered":"2024-04-26T04:32:12.503724+0000","last_unstale":"2024-04-26T04:34:33.531200+0000","last_undegraded":"2024-04-26T04:34:33.531200+0000","last_fullsized":"2024-04-26T04:34:33.531200+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:34:15.447891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:34:15.447891+0000","last_clean_scrub_stamp":"2024-04-26T04:34:15.447891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:35.945306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":231,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T04:34:29.292243+0000","last_change":"2024-04-26T04:34:12.649437+0000","last_active":"2024-04-26T04:34:29.292243+0000","last_peered":"2024-04-26T04:34:29.292243+0000","last_clean":"2024-04-26T04:34:29.292243+0000","last_became_active":"2024-04-26T04:32:12.597348+0000","last_became_peered":"2024-04-26T04:32:12.597348+0000","last_unstale":"2024-04-26T04:34:29.292243+0000","last_undegraded":"2024-04-26T04:34:29.292243+0000","last_fullsized":"2024-04-26T04:34:29.292243+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:34:12.649215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:34:12.649215+0000","last_clean_scrub_stamp":"2024-04-26T04:34:12.649215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:22.878039+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":233,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-26T04:34:29.169845+0000","last_change":"2024-04-26T04:34:18.368443+0000","last_active":"2024-04-26T04:34:29.169845+0000","last_peered":"2024-04-26T04:34:29.169845+0000","last_clean":"2024-04-26T04:34:29.169845+0000","last_became_active":"2024-04-26T04:32:12.702796+0000","last_became_peered":"2024-04-26T04:32:12.702796+0000","last_unstale":"2024-04-26T04:34:29.169845+0000","last_undegraded":"2024-04-26T04:34:29.169845+0000","last_fullsized":"2024-04-26T04:34:29.169845+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:34:18.368180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:34:18.368180+0000","last_clean_scrub_stamp":"2024-04-26T04:34:18.368180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:35:31.847787+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":467488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":9,"up_from":23,"seq":98784247842,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33152,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2019328,"data_stored":1179088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":23,"seq":98784247843,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33152,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2019328,"data_stored":1179088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":23,"seq":98784247842,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33672,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338168,"statfs":{"total":96636764160,"available":96602284032,"internally_reserved":0,"allocated":2551808,"data_stored":1708016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,9,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37296,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":2002944,"data_stored":1169444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313249,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2002944,"data_stored":1169444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2002944,"data_stored":1169444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,4,6,7,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":21,"seq":90194313249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28980,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342860,"statfs":{"total":96636764160,"available":96607088640,"internally_reserved":0,"allocated":2002944,"data_stored":1169444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,3,4,5,6,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":21,"seq":90194313249,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":1956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":2002944,"data_stored":1169444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,4,6,7,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247843,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33672,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338168,"statfs":{"total":96636764160,"available":96602284032,"internally_reserved":0,"allocated":2551808,"data_stored":1708016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,11,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280545,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28992,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342848,"statfs":{"total":96636764160,"available":96607076352,"internally_reserved":0,"allocated":2019328,"data_stored":1179088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,9,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37312,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334528,"statfs":{"total":96636764160,"available":96598556672,"internally_reserved":0,"allocated":2019328,"data_stored":1179088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[3,4,5,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":23,"seq":98784247842,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33152,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2019328,"data_stored":1179088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":23,"seq":98784247843,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2019328,"data_stored":1179088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247841,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2019328,"data_stored":1179088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,4,5,7,8,9,10,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37312,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334528,"statfs":{"total":96636764160,"available":96598556672,"internally_reserved":0,"allocated":2019328,"data_stored":1179088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,3,4,6,8,9,10,11,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247843,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2019328,"data_stored":1179088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:34:38.476 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T04:34:38.812 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T04:34:38.812 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T04:34:38.812 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T04:34:38.812 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:38.905 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T04:34:38.905 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:39.004 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T04:34:39.005 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-04-26T04:34:39.005 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:39.095 INFO:tasks.ceph.osd.8:Stopped 2024-04-26T04:34:39.096 DEBUG:tasks.ceph.osd.12:waiting for process to exit 2024-04-26T04:34:39.096 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:45.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.12 is failed for ~0s 2024-04-26T04:34:45.198 INFO:tasks.ceph.osd.12:Stopped 2024-04-26T04:34:45.198 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T04:34:45.199 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:45.271 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T04:34:45.271 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T04:34:45.271 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:45.355 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T04:34:45.355 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2024-04-26T04:34:45.355 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:45.441 INFO:tasks.ceph.osd.9:Stopped 2024-04-26T04:34:45.441 DEBUG:tasks.ceph.osd.13:waiting for process to exit 2024-04-26T04:34:45.441 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:45.522 INFO:tasks.ceph.osd.13:Stopped 2024-04-26T04:34:45.523 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T04:34:45.523 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:45.605 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T04:34:45.605 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T04:34:45.605 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:51.201 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-04-26T04:34:51.201 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.12 has been restored 2024-04-26T04:34:51.707 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T04:34:51.708 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2024-04-26T04:34:51.708 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:51.790 INFO:tasks.ceph.osd.10:Stopped 2024-04-26T04:34:51.791 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2024-04-26T04:34:51.791 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:51.853 INFO:tasks.ceph.osd.14:Stopped 2024-04-26T04:34:51.853 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T04:34:51.853 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:51.948 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T04:34:51.949 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T04:34:51.949 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:52.028 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T04:34:52.029 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2024-04-26T04:34:52.029 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:52.130 INFO:tasks.ceph.osd.11:Stopped 2024-04-26T04:34:52.130 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2024-04-26T04:34:52.130 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:52.220 INFO:tasks.ceph.osd.15:Stopped 2024-04-26T04:34:52.220 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T04:34:52.220 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T04:34:52.220 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:52.248 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T04:34:52.248 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T04:34:52.248 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:52.281 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T04:34:52.281 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T04:34:52.281 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T04:34:52.281 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:52.308 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T04:34:52.308 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T04:34:52.308 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:52.334 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T04:34:52.334 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T04:34:52.334 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T04:34:52.360 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T04:34:52.360 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T04:34:52.360 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | head -n 1 2024-04-26T04:34:52.378 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T04:34:52.379 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T04:34:52.580 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T04:34:52.580 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T04:34:52.752 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T04:34:52.752 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2024-04-26T04:34:52.869 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-12 on ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T04:34:52.869 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-12 2024-04-26T04:34:52.996 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-26T04:34:52.997 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T04:34:53.215 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-26T04:34:53.215 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T04:34:53.369 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-26T04:34:53.369 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2024-04-26T04:34:53.518 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-13 on ubuntu@smithi113.front.sepia.ceph.com 2024-04-26T04:34:53.518 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-13 2024-04-26T04:34:53.671 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-26T04:34:53.671 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T04:34:53.812 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-26T04:34:53.812 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T04:34:54.013 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-26T04:34:54.013 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2024-04-26T04:34:54.171 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-14 on ubuntu@smithi137.front.sepia.ceph.com 2024-04-26T04:34:54.171 DEBUG:teuthology.orchestra.run.smithi137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-14 2024-04-26T04:34:54.346 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-26T04:34:54.347 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T04:34:54.504 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-26T04:34:54.504 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T04:34:54.699 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-26T04:34:54.699 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2024-04-26T04:34:54.874 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-15 on ubuntu@smithi178.front.sepia.ceph.com 2024-04-26T04:34:54.875 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-15 2024-04-26T04:34:55.033 INFO:tasks.ceph:Archiving mon data... 2024-04-26T04:34:55.034 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/data/mon.a.tgz 2024-04-26T04:34:55.034 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-04-26T04:34:55.040 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.wWNEuf1Qx9 2024-04-26T04:34:55.041 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.wWNEuf1Qx9 2024-04-26T04:34:55.186 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.wWNEuf1Qx9 2024-04-26T04:34:55.254 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.wWNEuf1Qx9 is 412KB 2024-04-26T04:34:55.319 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.wWNEuf1Qx9 2024-04-26T04:34:55.326 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/data/mon.b.tgz 2024-04-26T04:34:55.326 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-04-26T04:34:55.333 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.KUhh7iA90d 2024-04-26T04:34:55.333 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.KUhh7iA90d 2024-04-26T04:34:55.516 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.KUhh7iA90d 2024-04-26T04:34:55.581 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.KUhh7iA90d is 583KB 2024-04-26T04:34:55.647 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.KUhh7iA90d 2024-04-26T04:34:55.653 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/data/mon.c.tgz 2024-04-26T04:34:55.653 DEBUG:teuthology.orchestra.run.smithi137:> mktemp 2024-04-26T04:34:55.659 INFO:teuthology.orchestra.run.smithi137.stdout:/tmp/tmp.tfIje3fsIQ 2024-04-26T04:34:55.659 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.tfIje3fsIQ 2024-04-26T04:34:55.841 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 0666 /tmp/tmp.tfIje3fsIQ 2024-04-26T04:34:55.906 DEBUG:teuthology.orchestra.remote:smithi137:/tmp/tmp.tfIje3fsIQ is 583KB 2024-04-26T04:34:55.971 DEBUG:teuthology.orchestra.run.smithi137:> rm -fr /tmp/tmp.tfIje3fsIQ 2024-04-26T04:34:55.977 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T04:34:55.977 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T04:34:55.980 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T04:34:55.982 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T04:34:56.022 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T04:34:56.042 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T04:34:56.048 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T04:34:56.081 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T04:34:56.088 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T04:34:56.088 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T04:34:56.090 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/remote/smithi012/crash 2024-04-26T04:34:56.091 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T04:34:56.105 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/remote/smithi113/crash 2024-04-26T04:34:56.106 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T04:34:56.119 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/remote/smithi137/crash 2024-04-26T04:34:56.120 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T04:34:56.137 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/remote/smithi178/crash 2024-04-26T04:34:56.138 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T04:34:56.151 INFO:tasks.ceph:Compressing logs... 2024-04-26T04:34:56.152 DEBUG:teuthology.orchestra.run.smithi012:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T04:34:56.154 DEBUG:teuthology.orchestra.run.smithi113:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T04:34:56.167 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T04:34:56.172 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30620.log 2024-04-26T04:34:56.174 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32391.log 2024-04-26T04:34:56.174 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30620.log.gz 2024-04-26T04:34:56.174 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30998.log 2024-04-26T04:34:56.184 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34878.log 2024-04-26T04:34:56.185 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32391.log: /var/log/ceph/ceph-client.admin.30998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33514.log 2024-04-26T04:34:56.185 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32391.log.gz 2024-04-26T04:34:56.185 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30998.log.gz 2024-04-26T04:34:56.185 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34878.log.gz 2024-04-26T04:34:56.185 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35112.log 2024-04-26T04:34:56.185 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29650.log 2024-04-26T04:34:56.185 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33514.log.gz 2024-04-26T04:34:56.185 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30089.log 2024-04-26T04:34:56.186 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35112.log.gz 2024-04-26T04:34:56.186 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29650.log.gz 2024-04-26T04:34:56.186 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29641.log 2024-04-26T04:34:56.186 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30920.log 2024-04-26T04:34:56.186 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30089.log.gz 2024-04-26T04:34:56.186 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31346.log 2024-04-26T04:34:56.186 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29641.log.gz 2024-04-26T04:34:56.186 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29639.log 2024-04-26T04:34:56.187 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30920.log.gz 2024-04-26T04:34:56.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31054.log 2024-04-26T04:34:56.187 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31346.log.gz 2024-04-26T04:34:56.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29639.log.gz 2024-04-26T04:34:56.187 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T04:34:56.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31054.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31054.log.gz 2024-04-26T04:34:56.187 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35643.log 2024-04-26T04:34:56.187 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32290.log 2024-04-26T04:34:56.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.35643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35060.log 2024-04-26T04:34:56.187 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35643.log.gz 2024-04-26T04:34:56.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31805.log 2024-04-26T04:34:56.187 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32290.log.gz 2024-04-26T04:34:56.187 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29670.log 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35060.log.gz 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31805.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31805.log.gz 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34694.log 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32392.log 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29670.log.gz 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34694.log.gz 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32734.log 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36516.log 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32392.log.gz 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29655.log 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32734.log.gz 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32992.log 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36516.log.gz 2024-04-26T04:34:56.188 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29655.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29655.log.gz 2024-04-26T04:34:56.189 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34480.log 2024-04-26T04:34:56.189 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30047.log 2024-04-26T04:34:56.189 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32992.log.gz 2024-04-26T04:34:56.189 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34480.log: 93.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-26T04:34:56.189 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34480.log.gz 2024-04-26T04:34:56.189 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32616.log 2024-04-26T04:34:56.190 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T04:34:56.198 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T04:34:56.198 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T04:34:56.198 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T04:34:56.198 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T04:34:56.198 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2024-04-26T04:34:56.198 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.13.log 2024-04-26T04:34:56.198 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T04:34:56.199 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.13.log: 86.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T04:34:56.199 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mon.b.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T04:34:56.202 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-04-26T04:34:56.209 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T04:34:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T04:34:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.14.log 2024-04-26T04:34:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T04:34:56.217 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-osd.14.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T04:34:56.218 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T04:34:56.218 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.10.log 2024-04-26T04:34:56.218 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-osd.10.log: 86.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T04:34:56.218 INFO:teuthology.orchestra.run.smithi137.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T04:34:56.243 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35138.log 2024-04-26T04:34:56.243 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30047.log.gz 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32616.log.gz 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36568.log 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35138.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35138.log.gz 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34904.log 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36568.log.gz 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36125.log 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30972.log 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34904.log.gz 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36125.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36125.log.gz 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30450.log 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31564.log 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30972.log.gz 2024-04-26T04:34:56.244 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32821.log 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30450.log.gz 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35242.log 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31564.log.gz 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35268.log 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32821.log.gz 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34450.log 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35242.log.gz 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35268.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35268.log.gz 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29673.log 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37426.log/var/log/ceph/ceph-client.admin.34450.log: 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34450.log.gz 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29673.log.gz 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34540.log 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34212.log 2024-04-26T04:34:56.245 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37426.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37426.log.gz 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36100.log 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34540.log.gz 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30104.log 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34212.log.gz 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36100.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36100.log.gz 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37192.log 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36060.log 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30104.log.gz 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37192.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37192.log.gz 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35664.log 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35034.log 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36060.log.gz 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34587.log 2024-04-26T04:34:56.246 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35664.log.gz 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30562.log 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35034.log.gz 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33466.log 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34587.log.gz 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36123.log 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30562.log.gz 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32207.log 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33466.log.gz 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36034.log 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36123.log.gz 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30094.log 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32207.log.gz 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36034.log.gz 2024-04-26T04:34:56.247 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35190.log 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29666.log 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30094.log.gz 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32689.log 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35190.log.gz 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29666.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29666.log.gz 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36066.log 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32689.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32689.log.gz 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31024.log 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36066.log.gz 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36490.log 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30130.log 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31024.log.gz 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36490.log.gz 2024-04-26T04:34:56.248 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30623.log 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34639.log 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30130.log.gz 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30623.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30623.log.gz 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37452.log 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35164.log 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34639.log.gz 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37452.log.gz 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31889.log 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37166.log 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35164.log.gz 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31889.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31889.log.gz 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34643.log 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36698.log 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37166.log.gz 2024-04-26T04:34:56.249 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34643.log.gz 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37010.log 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33607.log 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36698.log.gz 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37010.log.gz 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34038.log 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30868.log 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33607.log.gz 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34038.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34038.log.gz 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37348.log 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29672.log 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30868.log.gz 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37348.log.gz 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T04:34:56.250 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29588.log 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29672.log.gz 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33086.log 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29588.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29588.log.gz 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37322.log 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32660.log 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33086.log.gz 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37244.log 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37322.log.gz 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32660.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.33026.log -- replaced with /var/log/ceph/ceph-client.admin.32660.log.gz 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37244.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37244.log.gzgzip 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.31671.log 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36776.log 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33026.log.gz 2024-04-26T04:34:56.251 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35644.log 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31671.log.gz 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25103.log 0.0% 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36776.log.gz 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37114.log 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35644.log.gz 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.25103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36024.log 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25103.log.gz 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37036.log 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37114.log.gz 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36024.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36024.log.gz 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35580.log 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35629.log 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37036.log.gz 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29998.log 2024-04-26T04:34:56.252 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35580.log.gz 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35991.log 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35629.log.gz 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29998.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29998.log.gzgzip 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.30526.log 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35991.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35991.log.gz 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33264.log 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29536.log 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30526.log.gz 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35633.log 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33264.log.gz 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29536.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.29536.log.gz 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34726.log 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35633.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.35633.log.gz /var/log/ceph/ceph-client.admin.31931.log 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-04-26T04:34:56.253 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35649.log 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34726.log.gz 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31431.log 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31931.log.gz 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35294.log 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35649.log.gz 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31431.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31431.log.gzgzip 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33863.log 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30596.log 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35294.log.gz 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33863.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.33863.log.gz -- 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr: /var/log/ceph/ceph-client.admin.32045.log 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30596.log.gz 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33540.log 2024-04-26T04:34:56.254 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36438.log 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32045.log.gz 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36464.log 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33540.log.gz 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31101.log 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36438.log.gz 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36464.log.gz 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37528.log 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37140.log 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31101.log.gz 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31245.log 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37528.log.gz 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35634.log 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37140.log.gz 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31245.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31245.log.gz 2024-04-26T04:34:56.255 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34982.log 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35216.log 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35634.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35634.log.gz 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37400.log 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34982.log.gz 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35216.log.gz 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29674.log 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35398.log 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37400.log.gz 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29562.log 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29674.log.gz 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35086.log 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35398.log.gz 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35450.log 2024-04-26T04:34:56.256 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29562.log.gz 2024-04-26T04:34:56.257 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36724.log 2024-04-26T04:34:56.257 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35086.log.gz 2024-04-26T04:34:56.257 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35660.log 2024-04-26T04:34:56.257 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35450.log.gz 2024-04-26T04:34:56.257 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35666.log 2024-04-26T04:34:56.257 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36724.log.gz 2024-04-26T04:34:56.257 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30474.log 2024-04-26T04:34:56.257 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35660.log.gz 2024-04-26T04:34:56.257 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36750.log 2024-04-26T04:34:56.257 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35666.log.gz 2024-04-26T04:34:56.257 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33340.log 2024-04-26T04:34:56.257 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30474.log.gz 2024-04-26T04:34:56.257 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30091.log 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36750.log.gz 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30064.log 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33340.log.gz 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34155.log 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30091.log.gz 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33082.log 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30064.log.gz 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30446.log 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34155.log: /var/log/ceph/ceph-client.admin.33082.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34155.log.gz 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33082.log.gz 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34237.log 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30446.log.gz 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35665.log 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36672.log 2024-04-26T04:34:56.258 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34237.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34237.log.gz 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31779.log 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35665.log.gz 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36620.log 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36672.log.gz 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31779.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.31779.log.gz 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr: /var/log/ceph/ceph-client.admin.36854.log 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37218.log 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36620.log.gz 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36594.log 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36854.log.gz 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37218.log.gz 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36059.log 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32159.log 2024-04-26T04:34:56.259 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36594.log.gz 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32863.log 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36059.log.gz 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36984.log 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32159.log.gz 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32863.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32863.log.gz 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36072.log 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31728.log 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36984.log.gz 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36932.log 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36072.log.gz 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31096.log 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31728.log.gz 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29657.log 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36932.log.gz 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35008.log 2024-04-26T04:34:56.260 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31096.log.gz 2024-04-26T04:34:56.261 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34930.log 2024-04-26T04:34:56.261 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29657.log.gz 2024-04-26T04:34:56.261 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30525.log 2024-04-26T04:34:56.261 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35008.log.gz 2024-04-26T04:34:56.261 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30564.log 2024-04-26T04:34:56.261 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34930.log.gz 2024-04-26T04:34:56.261 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32155.log 2024-04-26T04:34:56.261 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30525.log.gz 2024-04-26T04:34:56.261 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32499.log 2024-04-26T04:34:56.261 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30564.log.gz 2024-04-26T04:34:56.261 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34725.log 2024-04-26T04:34:56.262 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32155.log.gz 2024-04-26T04:34:56.262 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31571.log 2024-04-26T04:34:56.262 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32499.log.gz 2024-04-26T04:34:56.283 INFO:teuthology.orchestra.run.smithi113.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T04:34:56.299 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30894.log 2024-04-26T04:34:56.300 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34725.log.gz 2024-04-26T04:34:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34352.log 2024-04-26T04:34:56.300 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31571.log.gz 2024-04-26T04:34:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30894.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30894.log.gz 2024-04-26T04:34:56.300 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.32071.log 2024-04-26T04:34:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35346.log 2024-04-26T04:34:56.300 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34352.log.gz 2024-04-26T04:34:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34826.log 2024-04-26T04:34:56.300 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32071.log.gz 2024-04-26T04:34:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34852.log 2024-04-26T04:34:56.300 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35346.log.gz 2024-04-26T04:34:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34826.log.gz 2024-04-26T04:34:56.300 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33544.log 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30574.log 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34852.log.gz 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33544.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.20156.log 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33544.log.gz 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33867.log 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30574.log.gz 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.tmp-client.admin.20156.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.20156.log.gz 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31093.log 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33867.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33867.log.gz 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30043.log 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31093.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31093.log.gz 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32528.log 2024-04-26T04:34:56.301 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30043.log.gz 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33702.log 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33806.log 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32528.log.gz 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33702.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33702.log.gz 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30065.log 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.33806.log: -- /var/log/ceph/ceph-client.admin.24972.log 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33806.log.gz 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30065.log: 86.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35502.log 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30065.log.gz 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.24972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24972.log.gz 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33056.log 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32232.log 2024-04-26T04:34:56.302 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35502.log.gz 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36958.log 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33056.log.gz 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31675.log 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32232.log.gz 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34457.log 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36958.log.gz 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37062.log 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31675.log.gz 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25235.log 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34457.log.gz 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33915.log 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37062.log.gz 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.25235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35636.log 2024-04-26T04:34:56.303 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25235.log.gz 2024-04-26T04:34:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30086.log 2024-04-26T04:34:56.304 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33915.log.gz 2024-04-26T04:34:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33431.log 2024-04-26T04:34:56.304 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35636.log.gz 2024-04-26T04:34:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.30086.log: -- /var/log/ceph/ceph-client.admin.36038.log 2024-04-26T04:34:56.304 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30086.log.gz 2024-04-26T04:34:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36071.log 2024-04-26T04:34:56.304 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33431.log.gz 2024-04-26T04:34:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29510.log 2024-04-26T04:34:56.304 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36038.log.gz 2024-04-26T04:34:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34129.log 2024-04-26T04:34:56.304 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36071.log.gz 2024-04-26T04:34:56.304 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32529.log 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29510.log.gz 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32016.log 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34129.log.gz 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37504.log 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32529.log.gz 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35630.log 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32016.log.gz 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37504.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37504.log.gz 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36906.log 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35630.log.gz 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32320.log 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33165.log 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36906.log.gz 2024-04-26T04:34:56.305 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.32320.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35642.log 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32320.log.gz 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30946.log 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33165.log.gz 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36802.log 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35642.log.gz 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29658.log 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30946.log.gz 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30498.log 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36802.log.gz 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29658.log.gz 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37374.log 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31436.log 2024-04-26T04:34:56.306 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30498.log.gz 2024-04-26T04:34:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37478.log 2024-04-26T04:34:56.307 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37374.log.gz 2024-04-26T04:34:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30052.log 2024-04-26T04:34:56.307 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31436.log.gz 2024-04-26T04:34:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30593.log 2024-04-26T04:34:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37478.log.gz 2024-04-26T04:34:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30052.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30052.log.gz 2024-04-26T04:34:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30080.log 2024-04-26T04:34:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34613.log 2024-04-26T04:34:56.307 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30593.log.gz 2024-04-26T04:34:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30028.log 2024-04-26T04:34:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30080.log.gz 2024-04-26T04:34:56.307 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33336.log 2024-04-26T04:34:56.308 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34613.log.gz 2024-04-26T04:34:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31382.log 2024-04-26T04:34:56.308 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30028.log.gz 2024-04-26T04:34:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33298.log 2024-04-26T04:34:56.308 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33336.log.gz 2024-04-26T04:34:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32860.log 2024-04-26T04:34:56.308 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31382.log.gz 2024-04-26T04:34:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29634.log 2024-04-26T04:34:56.308 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33298.log.gz 2024-04-26T04:34:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35476.log 2024-04-26T04:34:56.308 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32860.log.gz 2024-04-26T04:34:56.308 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32795.log 2024-04-26T04:34:56.308 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29634.log.gz 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32316.log 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35476.log.gz 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32795.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.12.log 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32795.log.gz 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36049.log 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32316.log.gz 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.12.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29653.log 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33780.log 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36049.log.gz 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36880.log 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29653.log.gz 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37296.log 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33780.log.gz 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30571.log 2024-04-26T04:34:56.309 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36880.log.gz 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29656.log 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37296.log.gz 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31832.log 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30571.log.gz 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33976.log 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29656.log.gz 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31316.log 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31832.log.gz 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33190.log 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33976.log.gz 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37088.log 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31316.log.gz 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.33190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29459.log 2024-04-26T04:34:56.310 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33190.log.gz 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35372.log 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37088.log.gz 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32690.log 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29459.log.gz 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33647.log 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35372.log.gz 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37270.log 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32690.log.gz 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.33647.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36010.log 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33647.log.gz 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.37270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36047.log 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37270.log.gz 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30573.log 2024-04-26T04:34:56.311 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36010.log.gz 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36047.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36047.log.gz 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34101.log 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30573.log.gz 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30563.log 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr:gzip/var/log/ceph/ceph-client.admin.34101.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34774.log 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34101.log.gz 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.30563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31498.log 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30563.log.gz 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34800.log 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34774.log: gzip/var/log/ceph/ceph-client.admin.31498.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29652.log 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31498.log.gz 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34774.log.gz 2024-04-26T04:34:56.312 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34800.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34800.log.gz 2024-04-26T04:34:56.313 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32855.log 2024-04-26T04:34:56.313 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35554.log 2024-04-26T04:34:56.313 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29652.log.gz 2024-04-26T04:34:56.313 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34426.log 2024-04-26T04:34:56.313 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32855.log.gz 2024-04-26T04:34:56.313 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35554.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35554.log.gz 2024-04-26T04:34:56.313 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36065.log 2024-04-26T04:34:56.313 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29660.log 2024-04-26T04:34:56.313 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.34426.log: /var/log/ceph/ceph-client.admin.36065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35661.log 2024-04-26T04:34:56.313 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36065.log.gz 2024-04-26T04:34:56.313 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34426.log.gz 2024-04-26T04:34:56.313 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29660.log.gz 2024-04-26T04:34:56.313 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35658.log 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29484.log 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35661.log: /var/log/ceph/ceph-client.admin.35658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36542.log 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35661.log.gz 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35658.log.gz 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.29484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29484.log.gz 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36828.log 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35528.log 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36542.log: /var/log/ceph/ceph-client.admin.36828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35657.log 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36542.log.gz 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36828.log.gz 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35528.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35528.log.gz 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34315.log 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34956.log 2024-04-26T04:34:56.314 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35657.log: /var/log/ceph/ceph-client.admin.34315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35424.log 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35657.log.gz 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34315.log.gz/var/log/ceph/ceph-client.admin.34956.log: 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29900.log 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34956.log.gz 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31102.log 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35424.log: /var/log/ceph/ceph-client.admin.29900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32575.log 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35424.log.gz 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29900.log.gz 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.31102.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.31102.log.gz 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.30046.log 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35320.log 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32575.log: /var/log/ceph/ceph-client.admin.30046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36646.log 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32575.log.gz 2024-04-26T04:34:56.315 INFO:teuthology.orchestra.run.smithi012.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30046.log.gz 2024-04-26T04:34:56.316 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.35320.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32123.log 2024-04-26T04:34:56.316 INFO:teuthology.orchestra.run.smithi012.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35320.log.gz 2024-04-26T04:34:56.316 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.36646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36646.log.gz 2024-04-26T04:34:56.316 INFO:teuthology.orchestra.run.smithi012.stderr:/var/log/ceph/ceph-client.admin.32123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32123.log.gz 2024-04-26T04:34:56.342 INFO:teuthology.orchestra.run.smithi137.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T04:34:56.462 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25154.log 2024-04-26T04:34:56.463 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25099.log 2024-04-26T04:34:56.464 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25250.log 2024-04-26T04:34:56.464 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25154.log: /var/log/ceph/ceph-client.admin.25099.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25099.log.gz 2024-04-26T04:34:56.464 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25154.log.gz 2024-04-26T04:34:56.464 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T04:34:56.466 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25178.log 2024-04-26T04:34:56.466 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25250.log.gz 2024-04-26T04:34:56.466 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25075.log 2024-04-26T04:34:56.470 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24979.log 2024-04-26T04:34:56.470 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25178.log.gz 2024-04-26T04:34:56.470 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25075.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T04:34:56.470 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25075.log.gz 2024-04-26T04:34:56.470 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.24979.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.11.log 2024-04-26T04:34:56.470 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24979.log.gz 2024-04-26T04:34:56.470 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25123.log 2024-04-26T04:34:56.470 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.29308.log 2024-04-26T04:34:56.470 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25123.log.gz 2024-04-26T04:34:56.470 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.15.log 2024-04-26T04:34:56.471 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.0.29308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25299.log 2024-04-26T04:34:56.471 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.29308.log.gz 2024-04-26T04:34:56.471 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.15.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T04:34:56.475 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25003.log 2024-04-26T04:34:56.475 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25299.log.gz 2024-04-26T04:34:56.475 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25226.log 2024-04-26T04:34:56.475 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25003.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.25027.log 2024-04-26T04:34:56.475 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.25003.log.gz 2024-04-26T04:34:56.475 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25274.log 2024-04-26T04:34:56.475 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25226.log.gz 2024-04-26T04:34:56.475 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25051.log 2024-04-26T04:34:56.475 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25027.log.gz 2024-04-26T04:34:56.475 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24955.log 2024-04-26T04:34:56.475 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25274.log: /var/log/ceph/ceph-client.admin.25051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25323.log 2024-04-26T04:34:56.476 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25274.log.gz 2024-04-26T04:34:56.476 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25051.log.gz 2024-04-26T04:34:56.476 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.24955.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25202.log 2024-04-26T04:34:56.476 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/ceph-client.admin.24955.log.gz 2024-04-26T04:34:56.476 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25323.log.gz 2024-04-26T04:34:56.476 INFO:teuthology.orchestra.run.smithi012.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T04:34:56.477 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.25202.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25202.log.gz 2024-04-26T04:34:56.487 INFO:teuthology.orchestra.run.smithi137.stderr: 91.1% -- replaced with /var/log/ceph/ceph-osd.10.log.gz 2024-04-26T04:34:56.552 INFO:teuthology.orchestra.run.smithi178.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T04:34:56.671 INFO:teuthology.orchestra.run.smithi012.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2024-04-26T04:34:56.819 INFO:teuthology.orchestra.run.smithi178.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T04:34:56.834 INFO:teuthology.orchestra.run.smithi137.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T04:34:56.876 INFO:teuthology.orchestra.run.smithi178.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.15.log.gz 2024-04-26T04:34:56.983 INFO:teuthology.orchestra.run.smithi113.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T04:34:56.990 INFO:teuthology.orchestra.run.smithi137.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.14.log.gz 2024-04-26T04:34:57.004 INFO:teuthology.orchestra.run.smithi137.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T04:34:57.005 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-04-26T04:34:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:real 0m0.821s 2024-04-26T04:34:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:user 0m2.521s 2024-04-26T04:34:57.005 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m0.109s 2024-04-26T04:34:57.013 INFO:teuthology.orchestra.run.smithi113.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2024-04-26T04:34:57.040 INFO:teuthology.orchestra.run.smithi012.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.12.log.gz 2024-04-26T04:34:57.049 INFO:teuthology.orchestra.run.smithi012.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T04:34:57.057 INFO:teuthology.orchestra.run.smithi113.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T04:34:57.179 INFO:teuthology.orchestra.run.smithi113.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.13.log.gz 2024-04-26T04:34:57.180 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-04-26T04:34:57.180 INFO:teuthology.orchestra.run.smithi113.stderr:real 0m1.012s 2024-04-26T04:34:57.180 INFO:teuthology.orchestra.run.smithi113.stderr:user 0m3.443s 2024-04-26T04:34:57.181 INFO:teuthology.orchestra.run.smithi113.stderr:sys 0m0.144s 2024-04-26T04:34:57.282 INFO:teuthology.orchestra.run.smithi178.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T04:34:57.294 INFO:teuthology.orchestra.run.smithi012.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T04:34:57.295 INFO:teuthology.orchestra.run.smithi012.stderr: 2024-04-26T04:34:57.295 INFO:teuthology.orchestra.run.smithi012.stderr:real 0m1.140s 2024-04-26T04:34:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:user 0m3.520s 2024-04-26T04:34:57.296 INFO:teuthology.orchestra.run.smithi012.stderr:sys 0m0.186s 2024-04-26T04:34:57.352 INFO:teuthology.orchestra.run.smithi178.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.11.log.gz 2024-04-26T04:34:57.353 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-04-26T04:34:57.353 INFO:teuthology.orchestra.run.smithi178.stderr:real 0m1.153s 2024-04-26T04:34:57.353 INFO:teuthology.orchestra.run.smithi178.stderr:user 0m2.461s 2024-04-26T04:34:57.354 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m0.125s 2024-04-26T04:34:57.354 INFO:tasks.ceph:Archiving logs... 2024-04-26T04:34:57.354 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/remote/smithi012/log 2024-04-26T04:34:57.355 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T04:34:57.970 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/remote/smithi113/log 2024-04-26T04:34:57.971 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T04:34:58.434 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/remote/smithi137/log 2024-04-26T04:34:58.435 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T04:34:58.795 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/remote/smithi178/log 2024-04-26T04:34:58.796 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T04:34:59.166 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T04:34:59.175 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T04:34:59.175 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T04:34:59.178 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T04:34:59.181 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T04:34:59.183 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T04:34:59.289 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T04:34:59.298 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T04:34:59.299 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:34:59.301 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:34:59.304 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:34:59.306 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:34:59.312 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2024-04-26T04:34:59.312 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2024-04-26T04:34:59.312 INFO:teuthology.orchestra.run.smithi012.stdout:+hv01.front.sepi 67.205.162.81 3 u 50 64 77 0.138 -1.839 1.575 2024-04-26T04:34:59.313 INFO:teuthology.orchestra.run.smithi012.stdout:+hv02.front.sepi 63.231.80.2 3 u 49 64 77 0.077 +0.836 1.785 2024-04-26T04:34:59.313 INFO:teuthology.orchestra.run.smithi012.stdout:*hv03.front.sepi 74.6.168.72 3 u 47 64 77 0.096 -0.458 1.636 2024-04-26T04:34:59.313 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:34:59.315 INFO:teuthology.orchestra.run.smithi113.stdout: remote refid st t when poll reach delay offset jitter 2024-04-26T04:34:59.315 INFO:teuthology.orchestra.run.smithi113.stdout:============================================================================== 2024-04-26T04:34:59.315 INFO:teuthology.orchestra.run.smithi113.stdout:+hv01.front.sepi 67.205.162.81 3 u 51 64 77 0.201 -2.183 1.676 2024-04-26T04:34:59.315 INFO:teuthology.orchestra.run.smithi113.stdout:+hv02.front.sepi 63.231.80.2 3 u 48 64 77 0.127 +0.542 1.674 2024-04-26T04:34:59.315 INFO:teuthology.orchestra.run.smithi113.stdout:*hv03.front.sepi 74.6.168.72 3 u 47 64 77 0.124 -0.785 1.731 2024-04-26T04:34:59.315 INFO:teuthology.orchestra.run.smithi113.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:34:59.317 INFO:teuthology.orchestra.run.smithi137.stdout: remote refid st t when poll reach delay offset jitter 2024-04-26T04:34:59.317 INFO:teuthology.orchestra.run.smithi137.stdout:============================================================================== 2024-04-26T04:34:59.317 INFO:teuthology.orchestra.run.smithi137.stdout: hv01.front.sepi 67.205.162.81 3 u 53 64 77 0.063 -2.495 1.010 2024-04-26T04:34:59.317 INFO:teuthology.orchestra.run.smithi137.stdout:*hv02.front.sepi 63.231.80.2 3 u 55 64 77 0.086 -0.310 1.062 2024-04-26T04:34:59.317 INFO:teuthology.orchestra.run.smithi137.stdout: hv03.front.sepi 74.6.168.72 3 u 48 64 77 0.104 -0.008 1.811 2024-04-26T04:34:59.317 INFO:teuthology.orchestra.run.smithi137.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:34:59.320 INFO:teuthology.orchestra.run.smithi178.stdout: remote refid st t when poll reach delay offset jitter 2024-04-26T04:34:59.320 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================== 2024-04-26T04:34:59.320 INFO:teuthology.orchestra.run.smithi178.stdout:+hv01.front.sepi 67.205.162.81 3 u 46 64 77 0.085 -1.932 0.663 2024-04-26T04:34:59.320 INFO:teuthology.orchestra.run.smithi178.stdout:+hv02.front.sepi 63.231.80.2 3 u 47 64 77 0.126 +0.685 0.545 2024-04-26T04:34:59.320 INFO:teuthology.orchestra.run.smithi178.stdout:*hv03.front.sepi 74.6.168.72 3 u 46 64 77 0.083 -0.631 0.542 2024-04-26T04:34:59.320 INFO:teuthology.orchestra.run.smithi178.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-26T04:34:59.321 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T04:34:59.330 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T04:34:59.330 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T04:34:59.339 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T04:34:59.349 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T04:34:59.358 INFO:teuthology.task.internal:Duration was 835.670598 seconds 2024-04-26T04:34:59.358 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T04:34:59.367 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T04:34:59.368 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T04:34:59.370 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T04:34:59.372 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T04:34:59.374 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T04:34:59.413 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T04:34:59.413 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-04-26T04:34:59.413 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T04:34:59.458 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2024-04-26T04:34:59.459 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T04:34:59.469 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-04-26T04:34:59.469 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T04:34:59.479 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-04-26T04:34:59.479 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T04:34:59.489 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T04:34:59.490 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T04:34:59.502 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T04:34:59.516 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T04:34:59.522 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T04:34:59.554 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T04:34:59.554 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T04:34:59.570 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T04:34:59.582 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T04:34:59.594 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T04:34:59.653 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T04:34:59.667 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T04:34:59.667 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T04:34:59.679 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T04:34:59.693 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T04:34:59.706 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T04:34:59.720 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T04:34:59.731 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T04:34:59.734 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T04:34:59.738 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T04:34:59.743 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-04-26T04:34:59.747 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2024-04-26T04:34:59.751 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T04:34:59.759 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-04-26T04:34:59.772 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-04-26T04:34:59.792 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T04:34:59.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:34:59.809 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T04:34:59.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:34:59.817 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T04:34:59.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:34:59.821 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T04:34:59.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:34:59.838 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T04:34:59.849 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T04:34:59.850 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/remote/smithi012 2024-04-26T04:34:59.850 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T04:34:59.877 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/remote/smithi113 2024-04-26T04:34:59.877 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T04:34:59.898 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/remote/smithi137 2024-04-26T04:34:59.898 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T04:34:59.921 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-26_01:23:56-rados-wip-yuri2-testing-2024-04-24-0914-squid-distro-default-smithi/7674537/remote/smithi178 2024-04-26T04:34:59.922 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T04:34:59.942 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T04:34:59.942 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T04:34:59.944 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T04:34:59.947 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T04:34:59.962 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T04:34:59.987 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T04:34:59.999 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T04:34:59.999 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T04:35:00.009 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T04:35:00.010 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T04:35:00.012 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T04:35:00.014 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T04:35:00.016 INFO:teuthology.orchestra.run.smithi012.stdout: 658642 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 04:34 /home/ubuntu/cephtest 2024-04-26T04:35:00.017 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T04:35:00.018 INFO:teuthology.orchestra.run.smithi113.stdout: 658467 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 04:34 /home/ubuntu/cephtest 2024-04-26T04:35:00.020 INFO:teuthology.orchestra.run.smithi137.stdout: 658467 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 04:34 /home/ubuntu/cephtest 2024-04-26T04:35:00.035 INFO:teuthology.orchestra.run.smithi178.stdout: 658466 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 04:34 /home/ubuntu/cephtest 2024-04-26T04:35:00.037 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T04:35:00.050 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T04:35:00.163 INFO:teuthology.run:Summary data: description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} duration: 835.6705980300903 flavor: default owner: scheduled_yuriw@teuthology success: true 2024-04-26T04:35:00.164 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T04:35:00.252 INFO:teuthology.run:pass